Your horse is exhibiting one of the most common stumbling blocks riders come across.
A bicycle. A bike can't bend. So, when it goes around a corner, it has to lean in because it can't bend. When the bike leans in, it forces the rider to lean in too.
This is what is happening to you and your horse. Horses, like people, are either "right or left handed". That means they have a stronger side that they prefer to put their weight on. Like us most horses are "right handed". They will prefer to put their weight onto that right shoulder. On a bend/circle, this is what usually occurs;
The horse tries to go clockwise, but doesn't bend.
All his weight tries to drop onto that right shoulder
This makes it impossible to bend now.
He counter flexes(bends to the outside)
Even MORE weight it now dropped onto the inside shoulder
He starts collapsing onto the shoulder causing the circle to get smaller and smaller
Since he is leaning in, he causes YOU to also lean in dropping all of YOUR weight onto the inside shoulder even further complicating matters.
Sound familiar? I bet it does.
To start fixing this, you need to start at the walk. When you ask the horse to circle clockwise, keep your outside rein to stay dead steady. You want your horse to learn to trust that rein and desire to move their weight into it. Any unnecessary jerking or pulling on it will keep your horse from wanting to use it constructively.
Use your inside leg
to push push push the horse into the outside shoulder while using the inside rein to finish the bend, bringing the head inward. If you just use only the rein and no leg, you will only pull the horse back onto that inside shoulder. Occasionally, you might half halt with the inside hand to keep the horse from leaning on the inside rein.
When the horse can maintain this bend at the walk, start all over again at the trot. As the horse leans in, you can help him lighten the inside shoulder by taking your own weight out of the equation. Shift onto your outside seatbone/stirrup and put extra weight there. This will help him a LOT. Later, as he gets better at bending and arrying your weight, this will not be necessary.
Start with an energetic but SHORT trot stride. It is much easier for the horse to balance with a shorter trot stride. Energetic though, not like a western jog. Later, as the horse can maintain the bend and stay balanced, you can start lengthening the stride, but not too soon.
The bend always starts with the leg.
When he can carry himself well at the trot, both short and longer, then start all over with the canter. Again, keep the stride short, it helps. At the canter, the horse will really want to lean in. You will have to shift your weight heavily to the outside seatbone/stirrup at first, just to help him. Ask the canter depart from a SHORT trot stride. Every other stride, perform a half halt to keep his stride short enough that he can balance himself.
This is what I do with the countless horses I have had with this problem. It has worked every time. Good luck. I hope this helps you.