He is not your baby...he is a horse and I would imagine what you are seeing is the result of your babying him.
I seriously doubt there is any pain involved, just a horse that has decided to do what it wants. In other words, this horse is spoiled.
And fighting with him gives him a battleground.
Instead of battling, just wait it out by sitting on him with his head facing way you want to go.
And patting him and telling him good boy does nothing but make this worse. He has disobeyed, and you are praising him? For what?
Your handling skills on the ground are more than likely lacking, they usually go hand in hand with this.
Get a trainer to work with horse and you, but do not expect the horse to be cured, as any horse that knows it can run the show will do exactly that, and all horses will try this if given the chance.
Walking him home, and round penning him also does nothing except make this worse. He wanted to go home, round penning him is of no consequence since he won.
If you don't get this fixed, and learn how to deal with horses better, he may well flip over on you.
And do not be fooled into thinking as so many do, that you can just buy another horse and fix this, as every horse will do exactly as this one has learned to do if not handled correctly.
Horses make me a better person.