The stall guard is something he could get tangled in and will if he panics. Why not have him put out as the 2nd horse to go out. That way he's not alone in the field, nor alone in the barn. This is something your are going to have to work on or riding isn't going to work out. Something I suggest is taking him for a walk away from the others and as soon as he begins to get upset, turn back but make him mind his manners. When knows the others are nearby, take him back on his walk again, then return. Each time you will try to take him a little farther. Offer him a few treats before you turn back. This give him a pleasant association. What you are doing is creating an elevator effect. As you leave his stress goes up, on the return it goes down. His mind will want a release from the stress and he will get better at going away. Be prepared to spend an hour doing this and keep offering a treat when you reach the farthest point. Do this three days in a row if you can. I'll even stash a pan of feed just past the previous last point so he can see it. He'll begin to look forward to leaving the others. When he's ok with this, do the same when turning him out first, show him the pan of goodies and he'll soon want to be first one out.