Ticks, and the aural plaque mentioned are main things to consider. And make sure you do not allow him to make this a battle that he wins all the time. It may just be that no one has ever told him, yes I am touching your ears.
And as for ear twitching. It is not a bad thing per se.
I worked Saddlebreds and we used to ear twitch them and not a one of them was head shy either, not one.
We did not, however, use the "grab 'em, bend 'em and twist 'em off at the roots" method that you oftentimes see on TV.
We simply ran hand up neck, took hold of ear, and gently twisted, no bending, no pulling, just partial twist too. Slightly more twist if horse acting up. It worked perfectly.
And as I said, this was in training barn, and these horses were not head shy and did not mind having their ears handled at all. These were Saddlebred Show Horses, and very expensive, and were handled daily, as well as clipped, braided and put into tailsets.
For that matter, it was also routine to float teeth ourselves, and clean sheaths, without tranqing.
The problems come from someone not knowing how to ear twitch correctly, and using the wrong method.
Horses make me a better person.