I would rather have a rope halter on, keep the end of it over my arm (not around, just draped over, ) and if he starts to move, you could either do as Cherie suggests, snap the rope halter under his chin a good one and say WHoa! or hold onto the rope halter with your left hand and let him cirlce around you and the saddle until he choses to stop. YOu'd have to turn with him. If you had an English (lighter ) saddle would be so much easier.
I would try what Cherie suggested.
I just think you can do som much more with a rope halter and a longish lead than the bridle. Just my opinion.
I have't personally dealt wtih this problem to the level you have. If my horse moves, I will verbally scold him. If he chosed to move more, I just follow him .
If he moved into me I would smack him with whatever I could, of kick him in the belly if he seriously barged into me like he was going to run me over and I had no free hand and nothing to swing at him. I have feet and sharp cowboy boots.
OR, just stamp the ground fiercely, if that will change his mind. Acting big and angry for a second can really make an impression. Also , takes his mind off that saddle and onto the wacko stamping the ground right next to him.