it's possible that it is rockering on him. In ohter words, the opposite of bridging. you know what I mean? the place with most contact is right in the middle, so the front and the back "rocker".
try it on his back without any pad. put it way too far forward, like up on his neck practically, and shove it toward it's "place". as you get close to where you think it should go, slow down, and kind of tap it with the palm of your hand for the last few inches. tap lightly. if it needs to move more to find home, it will. once it settles into "home", you will feel more resistence to your light taps. Check the fit there.
reach your hand up under the fender and feel right up under the tree, running your hand from as far forward as you can reach to as far toward the back as you can reach (all under the fender/skirt, from standing at the side of your horse) . Does the contact of the tree to the horse feel even the whole way?