I just posted this in another thread, but it applies to yours, too, I think.
The horse needs to learn to bend!
The problem is that he is putting too much weight where it does not need to be.
He is acting like a bicycle going around a turn. It can't bend, so it has to lean to turn. When the horse leans, it weighs his inside shoulder making it even more difficult to bend. The lean throws the saddle ( and YOU) to the inside too. This adds your weight to the inside shoulder increasing the problem exponentially.
You have to be creative to help solve this. At the walk and trot, do lots of bending/suppling exercises like serpentines and turns through the circle.
When riding, for now, you cannot sit in the middle of your saddle! The saddle is leaning in with the horse. You must shift your weight to the outside, as you have been trying. This is only for now. Your upper body needs to be perpendicular to the ground not the leaning saddle. This will help bring some of all of that weight to the outside shoulder lessening the load on the horse's inside shoulder. It will help.
Then, as you work on the bend through inside leg and light inside rein, you can move the horse back under yourself. Since the horse is using speed to balance, you must use constant half halts to try to bring the horse back onto his haunch. You might need to half halt on every stride for a while.
Every once in a while, drop that inside rein to keep the horse from leaning on it to balance. They will often do that. Drop the rein for a couple of strides (between half halts) and then softly take it back.
As the horse starts bending, your upper body will now stay perpendicular to the ground. You will find that the saddle is no longer being thrown into the circle, and you will be in the middle of the saddle again.
Hope this helps.
Read more: Helping the horse balance at Canter