I think that what you've been doing is pretty good. Instead of pushing him to back up until he refuses, though, try just asking him for a step. Then praise, then turn him around, SLOWLY, pause before exiting trailer, and take him out. Then again. Only next time, ask for two steps. Then turn him around, SLOWLY, pause before exiting trailer, take him out. Repeat, this time with three steps backing. Then be done for the day.
Slowly but surely. I once had a horse that did almost the same exact things yours does, but mine had been tied in a trailer too tightly, and when he went to back out of the last stall, tired of holding his head up the whole time, his back feet made it, and slipped under the trailer, but his front couldn't go back anymore, and he was stuck. They left him until his rope eventually broke. GRRR, I just do not understand people like that. But anyway, he loads the best of all mine now, just took a little time to show him I wouldn't ever do that to him.
But anyway, time, patience, and understanding. Sounds like you're doing pretty good so far. Just a little bit at a time. You're not losing the battle if you only ask him for a little, and not asking until he refuses, because then YOU tell him when it's okay to turn around. Try using commands. I always say Back, back, back... And then when they get to the step down, I let go of all pressure and just say down, down, down... If they don't take the last step down then I just give them a tiny bit of pressure and keep saying down. To load, I just say Come on, get in. LOL. I think what you're doing so far is pretty much what you should be doing, but don't ask for it all at the same time. Good luck!
To rein a horse is not only to guide him,
but also to control his every movement.