You know, from my perspective.....I can't even get my horse to walk a straight ine in the arena! I am openly honest about my riding ability! Makes it easier to ask for help!
Your horse sounds 'busy' and that's ok
. What I would do, if it bothers you so much, is really work on him responding to your legs....think of him being in a straight line from the tip of his nose to the tip of his tail.....do nag him, nag him to be straight for a few strides or more than a few strides then let him relax, and the more he gets used to being straight, then ask a little more until it becomes second nature for him to go/walk where you guide him, and if that's straight, then so be it.
With a puppy dog type horse, a little nagging won't hurt....he needs to pay a little more attention to you.....he's a world observer