I would get him over his tiff with his ears being touched before you even try to clip them.
Spend a lot of time, just running your hands over them, and the reaction will get smaller. He'll eventually learn to like his ears being scratched and rubbed.
For future reference of clipping-Buy some small clippers made for the ears,face, legs, etc. They're small, fit nicely in the ear, don't get hot, etc. I wouldn't even try it with normal size clippers.
I made that mistake, and ended up cutting my horses ear, because the blades get hot to quickly when your trying to work with a horse who hates his ears clipped, and he tossed his head really hard, and got a big stinging cut by his ear. Big problem. My horse who loves his ears scratched, was then ear shy for a couple weeks. So, I got him loving his ears rubbed again, and I bought these-Wahl Touch Up Trimmer in Clippers at Schneider Saddlery
Adorable, and again, are easy for small areas and don't get hot.
It took about thirty minutes, of take and release, and my horse was standing there with his head almost on the ground leaning into the clippers enjoying the scratch of them, while I did his ears with no issue.
If you're going to keep your horses ears clipped, you may also want to invest in a fly mask with ears for the protection.