That's the heck of it, isn't it? So much is determined by context, yet horses don't contextualize. LOL
The best advice I have is go with intuition and instinct. Don't ignore it. It's there for a reason. If you honestly feel its affection in your gut, enjoy it and return the favor. If you feel you're being bullied or pushed around, there's an immediate physical danger: You're going to get smooshed against a wall or stepped on or worse (Let me tell you, getting your foot stood on by a hooved animal the size of a horse hurts bad enough you can't even articulate at the moment. You just make unintelligible utterings while shoving the beast off your foot as fast and hard as you can, and you can't move fast enough LOL). In a larger arc (Think of ripples in water when you cast a stone) yes, its the Who is in Charge Here question.
As someone who has a horse that's gotten spoiled over the summer by my kids and now crowds and pushes and even charges sometimes, I can tell you - you do NOT want an intelligent, 1100 lb herd animal being in charge.
And let me tell you the difference:
Superman crowds. His ears are backed, but not relaxed and swiveling as if listening, his body is tensing as if for a fight. He knows what he's doing and its quite deliberate. I feel challenged to push back. He's challenging me. I push back and we have a disagreement and I WILL win because its that, or let him be the boss, and I do not want that. No one does.
Conversely, Gina will seek me out, quietly walk up behind me while I'm watching the heifers and just watching the sunset and leaning against the fence. She will gently drape her head over my shoulder, or has in the past rooted her muzzle up under my arm so I'm hugging her head, and she presses in to me or pulls me to her - gently. And we stand there in companionable silence while I scratch or hug or pet her. Its all quite relaxed, no one feels the need to challenge or respond to a challenge.
You know the difference in your gut, even if you can't point out the specific reasons or body language. Listen to your gut.
"We are here to laugh at the odds and live our lives so well that death will tremble to take us."