Certainly wouldn't be typing a horse or holding directly onto their halter when you're worming them unless you know the horse extremely well and that they are good to worm. You can wind up with a horse with a brokwn neck, you with a broken hand/dislocated should etc.
As others have said, make worming a pleasurable experience for him. I definitely don't think it is valid to jump to the conclusion that he wasn't handled right previously, that's crazy, so many horses play up with worming... come on, as kids, who willingly took their medicine when it tasted terrible??? Of course the horse is going to put up a fight, some horses don't mind it so much, others can't stand it.
I use either applesauce, or watered down custard powder. Something sweet will do the job!! Fill and old worming syringe with whatever you chose to use, and 'worm' your horse daily with it. They'll figure out that it actually tastes pretty good and will end up wanting more. Of course when you worm them properly again they're going to be disgusted, but just keep on with the yummy stuff every few days and they'll learn to live with it.
ALWAYS hold onto a lead rope though when you worm!!!!!! Very important! Also, don't swing off his head, as tempting as it is when they throw they're head up and run backwards, don't pull down on the halter, they want to run backwards, so you run at him and force him backwards until he doesn't want to go back anymore, then back him some more. THEN try the wormer again. He goes backwards, same drill.
It's the same as all training, if they react in a negative manner, make it uncomfortable, if they react in a positive manner, make it comfortable. No dramas ;)