The only confidence he lacks is confidence in your ability to make him go a different direction than he would rather go. I, too, (or is it three), have not had much luck with working a horse at the gate. I've had a lot more positive response by not letting a horse go anywhere near the gate. I want him to know that I am in charge and as long as he acts anxious to get to the gate, we are not even going there. As soon as a horse speeds up going toward the gate, I just fold him around on the fence and go the other way. I have had horses that I made change directions a hundred times until they walked quietly, head down and relaxed toward the gate. Then, I ride them to the farthest corner and sit on them there for 5 minutes before I dismount, loosen the girth and lead them out.
If a horse acts anxious when I stop and stand at the far end, I will get off and tie it in the far corner -- maybe for hours -- until it is standing there relaxed and resting a hind leg. [This, of course, could be impossible in a 'public' arena or an arena made of portable panels, but it sure works here at home.]