I'll make sure to try that next time, thanks!
But I've been told that doing that might desensitize him to leg pressure and I'll end up with a horse that ignores my legs?
Only if you do it every time for a long period of time. Right now, you are doing it anyway by teaching him he can ignore your cues.
I have a young horse who was like this at first. He hadn't had a lot of schooling. I really had to get at him with legs, body and crop to get him to trot initially, and he would only trot a little bit, then go back to a walk. I had to go at him hard every time he did that until he figured out that he's supposed to stay at the trot until I tell him he can walk. It wasn't pretty at first (picture legs flailing, crop whacking his butt, voice urging him on), but now, just a little squeeze is enough to get him trotting and he actually seems happy to trot. He has more energy and impulsion too, and I don't have to keep squeezing all the time or keep asking. So he hasn't become desensitized to my leg, quite the opposite. He now understands that if he doesn't respond to a firm squeeze, I will go at him hard, so he gives it to me immediately now.
Don't get me wrong, I don't go at him hard enough to scare him, just hard enough that he knows I mean it. It's difficult to describe. Just remember, this is a 1000 lb animal. Tapping him like you'd swat a fly isn't enough.
Just the other day, an adult beginner came to ride so we put her on this guy because he's so safe. Sure enough, they stood there forever while she gently tried to "ask" him to move forward. It was hard not to laugh. She would tap her heels on him, flick the crop like she was afraid to hurt him... he didn't care at all. Forget trotting. My daughter had to walk in front of him the whole time to get him to move. After about half an hour of watching this painful "lesson", I decided to hop on him myself just to make sure he wasn't learning a bad habit of ignoring his rider. The minute I got on him, he walked on, and we trotted in both directions easily. I didn't even use the crop. Completely different horse.
This often happens with beginners. Once you get him to move forward, you'll figure out just how much energy you need to use. Keep at it!