If your horse is worried and submissive, he might be looking for a place of comfort. You want to be that place of comfort. In the round pen, you are being as calm and available to him as possible, yet he continues to look outside the round pen for that comfort.
What you may have to do is make him realize that out "there" is not so comfy, and in "here" is. So, when he looks outward, do something to interupt his outward thought. Just enough so he stops looking outward. He doesn't necessarily have to look at or come to YOU, but he has to stop looking outward.
. All this can be done with him just standing out there on the rail, or with him moving around at any speed. And the thing you do to interrupt his thought can be as small as a wave of one finger, a scuffle of your boot in the dirt, or a huge wacked out wave of a flag with you jumping up and down like a banshee. It depends on how hard he is focussed out there. Start small and go as big as necessary but stop INSTANTLY when he ceases his out ward focus.
Then, he will either chose to look at you, or look outward again . That is his choice. If he looks outward, simply interrupt that thought. Don't do it too quickly, however, when dealing with a timid horse. Give him a few seconds as he might change his mind and come back to you all on his own. And that is what we want; we want him to choose to find YOU more attractive than the outside., and sometimes that means make "outside" a place where he is irritated by these noises and such, inside where there is calm. If outside is calmer or more interesting than inside, he will never choose to come in.
Once he looks at you with both eyes, you can "invite" him in by turning your core body away a bit, maybe taking one or two slow steps backward, and if he gets stuck and can't move, you circle a bit around to his hindquarters.
If he really gets stuck and wont move, belevie it or not, you do the same "interrupting" motion you used before, big enough to move his feet and inspire him to make a choice; either move away or move toward you.
Keep insiisting he make choices until he choooses you. If he won't come up to you the first time but will stop and look at you, you can go up and pet him . This is especially useful for a timid horse. If you are too forceful with "make a choice" he will not realize that there IS any other choice but to flee.