I agree with Wallaby, to a point. If you are just walking by, with your intention very obviously focussed somewhere else, and the horse pins his ears at your passing, but does not act aggressively in any way, I might walk by and ignore it, as I am not requiring anything from the horse at all. But, if I am in ANY way dealing with that horse, such as walking up to him , not another her, HIM, trying to catch, lead or saddle him, I would NOT tolerate that for a sec.
You see, as my trainer told me, it's not just that he is expressing defensiveness or aggressiveness toward you, which is not acceptable or safe, it's that he has a very negative frame of mind. And if you do nothing, you LEAVE him in that negative space, and he does not have to be there. You aren't doing a single thing to hurt him. So, you should "help" him to leave that ugly mental space. I don't mean "beg" him by being nice, or lure him out with treats, but rather give do something that interrrupts that thought strong enough that the horse literally forgets he was being pissy, becuase YOU have suddenly become the focus of his thoughts, and not in a negative way, but more in a curious way.
For example, my friend's hosre is very girthy. Snappy ears, gnashing teeth, lowered and defensive head. This is from his anticipation of pain, from year of wearing an ill fitting saddle. The current saddle is an excellent fit. He does not have a sore back, as you can palpate him fine. But, his mental state becomes defensive and ugly by habit. My teacher would take the line and give it a good hard shake. The horse might throw up his head, and look off to the side. That's not enough. She might give the line another shake, or ust a wiggle, whatever it takes till that horse has to look down the line at her with a "what? Oh, you are there, and what would you like?" sort of expression. It's like the whole feeling of "I gotta pin my ears" is GONE from his face.
It looks like she is punishing him, but she stays in there until he gives up that ugly state of mind. THIS is what is important. The horse will be happier when he is not left in this same old nappy attitude every time he interacts with this human. And that is another reason to disallow that behavior.