You need to work on getting him relaxed and paying attention to you first. Only then should you ride him on the rail. For now I personally would just start riding him in the middle of the arena.
Before you even get on to ride just walk him around the arena, and even in the corners until you start to see him relaxed, it could take 5mins to half an hour. But if you do this everyday, everyday it will become less and less. It really does make the world of a difference. I did this with my horse for a month.
And once you can do the things I posted above, once you start riding on the rail, work him, work him, work him, and EVERY time you want to stop or "whoa" rest him in one of the corners. Then he will think "Oh everytime we stop in the corner I get the reward of rest, this isn't so bad".
Although I wholeheartedly agree that you need to get him relaxed first and foremost, if he's already diving into the corners I wouldn't purposefully rest him there. He'll only dive more enthusiastically if he feels like that's where he'll get to rest
To relax him, first make sure that you're relaxed. If you're tense, he will
mirror your tension. Keep up with the circles and serpentines; lots of transitions (changes of gait, direction, speed, or balance
) to get him focused on you. Be sure that you are riding the loops correctly. If you're not balanced, he can't relax because he's balancing for two. When he's relaxing, you'll feel his back swinging more freely, his head will settle with his poll a little higher than his withers. Feel/listen for his footfalls to get more rhythmic. Use your seat to influence the rhythm and ride the back end. You can do a profound amount of relaxation through your seat.
When you do ride the rail, start after he's relaxed. If he isn't relaxed, he isn't learning anything. Use your inside leg to encourage him to stay near the rail, your outside leg to support him. If you're riding English, don't use your outside rein to drag him to the rail, but do keep a light feel on it. You can raise your inside hand a bit if he starts diving away from the wall to keep his inside shoulder from collapsing. If you're confident to put the reins in one hand, you can rub his withers to reassure him that moving down the rail is a good thing. Start at the walk. If he can't walk calmly down the rail, speed isn't going to help any.
Is there any possibility that he has a vision problem? It's fairly common for vision-impaired horses to shy away from the rail, or exhibit similar difficulty relaxing. Just something else to rule out.
Best of luck!