if you post a photo we might be able to see if the saddle is not fitting correctly. If it is too wide in the front, it will sit down too far and make that downhill feeling worse. It could be a saddle issue, could be the conformation and lack of engagement of the hind, or a bit of both. Have you a photo of the saddle on him? Take some with no blanket underneath, from front, side, front angle (off shoulder) and rear. Stand him on flat ground and square him up.
then a shot of you in the saddle. If you have the time.
My Appy is built downhill and I find an english saddle on him makes me feel as you described and even hurts my lower back after an hour, 'cause it's pushing up into my back. But a western saddle helps to level that out and I can sit vertically without have to fight gravity.