Honestly, to make it better, practice doing it a lot, on the wall so you have a wall to support the straightness and also on the centerline. Make sure your body is really straight, not tipping to one side or pulling one shoulder back. If he consistently throws the hip to one side, close that rein to create a 'wall' to keep his hind end straight. I.e. if his hip swings to the right as you halt, close the right rein to keep it straight. Pushing them more forward helps as well, practice halting from a trot too, really push him forward and straight into the halt, make sure you and your horse are straight before asking for the halt.
Hope that helps.