Although I believe this is most likely a behavioral issue as other members have stated, I'd like to add my two cents. Many TB's are thin skinned and ticklish, making them very sensitive horses. Maybe you are grooming too lightly and tickling him?
One of my geldings went through a phase awhile back where he didn't like being groomed--I later realized it was the stiff dandy brush I was using. When I switched back to the softer brush, he would let me groom him without protest. This gelding could just be sensitive to grooming techniques and tools if manners aren't the issue.