We can't ask him what is wrong so it requires detective work. Try to figure out what is the common denominator each time he misbehaves...the same weather (windy and loud), the same saddle, the same time of day (close to feeding time), the same issue with arena setup with scary objects, etc.
This is not spoiling a horse. This is understanding that this horse is an individual, unique from every other horse. I believe it is much better to understand a horse's motivations rather than just pushing them through everything. Pushing them will get you through today. But if you do this a few times and a horse has legitimate reasons for his behavior (pain, fright, confusion over what you are asking) then he will start to believe that you will not listen to him.
From what I have experienced this creates horses that either begin to speak louder in hopes of being heard by making their behavior more drastic (rearing, bucking, etc.), or else become dull, or else begin to quickly throw out behaviors in hopes that this is what you are looking for so they avoid punishment. I've been on horses that have been trained that if you say "jump" they say "how high." When you ask gently to "stop" they will slam on the brakes. When you lightly ask them to turn they will throw themselves in that direction, even if it means hitting another horse. Loud people create loud horses.
I believe in this quote: "If your horse doesn't do what you say, he either doesn't understand the question or else you are asking the wrong question." Except I believe there is one more component to this which is the horse saying "I can't because ____." So we have to listen to horses too and not just order them around.
Say a horse refuses to go forward through a narrow space. You can either sit on the horse and spin and rear and back while he refuses, or else you can ask him three times and if he can't figure it out, get off and lead him through. For every horse I have tried this on, the horse did not think he "won," but instead realized he could go through that space and was more than willing to be ridden through the next time. It's thinking of a different way to pose the question to the horse.