My gelding holds his ears back when we ride with a bunch of other horses. We just started a new winter group ride thing at my barn, we get together 2-3 times a week and play with our horses. My horse is at the bottom of the herd, so all of the horses we ride with have at least some dominance over him.
I don't know if he is just unsure of himself around them and he is paying closer attention to me, or if he not paying enough attention to me and is just being grumpy about it. He pins them when another horse is coming up on his butt too, and I always tell him to chill out. He has never threatened to kick out at them, but he gets super irritated when they rush up on him.
When we ride alone, his ears are almost always forward and attentive. We have been riding alone most of the time for the past year, so maybe he is just spoiled by it. It has to be the other horses. Regardless, I think he just needs to trust me more and gain some confidence around them and learn to relax.
"A rider who would trade partnership for obedience
will have to settle for neither."