I would say it is time to ditch the side pull and train in a snaffle at least.
Work on gaining a one rein stop on him. Since he already responds to your seat, this may come fairly easy, its just you will take up one rein, to bring the halt rather than both. It sounds to me, you have worked on more verticle flexion than lateral flexion...time to back it up a bit and pick up the lateral flexion, and gain that peice of training that could save your butt on ANY horse you train.
"The ideal horseman has the courage of a lion, the patience of a saint, and the hands of a woman..."