My answer is, it depends. Obviously gently nudging didn't work, so I would say start with no leg pressure. Is this a horse that would have a melt down if you got into him with a sharp verbal and crop correction when he started to get sticky? I don't want to give advice that lands you somewhere in the next time zone. Some horses give right in to a sharp correction and keep going. Others do what yours does and laugh as the gallop home without you. If your horse is sensitive, just out-stubborn him. Using only reins, no leg pressure, keep him facing the direction that you want to go and sit there like you have all day. You may have to sit out numerous tantrums, but eventually he will figure out his only choice is forward. Once you feel his body start to relax, ask for forward with only reins via the drunken sailor walk. Use one rein and then the other to get him to follow his nose. The result will be a zig zaggy, but non-the-less forward walk. When he does go forward, again depending on his personality, either praise him low key with a simple release of aids, or throw an all out party with lots of pats and good boys. Good luck.
You just have to see your distance...you don't have to like it.