I think the "work hard near the barn" idea is good. ON top of that, if there is a place that is "out" where you can let the horse graze for a bit, that makes "out" even more appealing.
If you do need to get off ( no shame in that) then walk him forward, away from the barn and past the "sticky" spot, get back on and see if he will continue. Do this a couple of times, if he still wont' go forward, then go back to the barn and WORK!!! Then after the work, start out again. Let him turn around to go back, if he will. Just go along with him but once back at the barn , go back to WORK! After 5 minutes of HARD work, go out again, letting him choose whether to go back to the barn or go forward.
This can be very time consuming, so be sure you have several hours to accomplish this, because you must go all the way through it. You must continue to work him every time he chooses to go home . The first time he does not choose to go home at the "sticky" point, you ride him just a bit further (because you have won the battle), turn him around, take him home, detack and put him away.