Looks like he has a bit more back for a larger saddle. The western one is too small, you don't look comfy in it.
My heart goes out to you hun. It's so tricky to find a saddle that fits both you and the horse. I spent ages trying to find a decent English saddle as my girl has such a short back western saddles look stupid on her. When you do end up buying your own, get it fitted, just to make sure.
And I don't think you're doing too much damage atm as you're not running off on trails for hours and hours :)
Also, love that horse's butt ;)
I am her eyes. She is my wings. I am her voice. She is my spirit. I am her human. She is my horse.