A few things that jump out at me - swapping leads/not holding leads, taking off far away from fences, and any similar behaviors speak of a combination of lack of fitness as well as potential pain issues. Not to say that he's lame by any means, however older horses especially when out of shape and not in work, can be prone to back and hock issues. The less the horse is mobile with impulsion, the higher the chances of both arthritis as well as spine misalignment due to muscular atrophy putting uneven pressure on the bones (as the body weakens, we all favor one side over the other - this can put uneven pressure on the spine and bones causing chiropractic issues).
To start, lots and lots of walking is a huge help. Hill work will help build muscle - keep him at the walk for now - and help his joints loosen. After several consistent all walk workouts of 30 minutes each at least, then add a few minutes of trot. Hills are still great for this as they will help the horse use their hind end more. Start with trotting uphill, and walking down hill, but eventually trotting both up and down will help. Alternate workouts with some longing in varying circle sizes, just take it slow as longing can put excess strain on the joints.
As he builds up more fitness, gradually add in canter work - with only a few minutes per workout. Monitor his breathing as well as look for any stiffness to ensure that he's able to canter without issue. If you can have a chiro come out, it's great to have done before cantering as the trot work will build the muscle and help hold the bones in proper alignment.
From there you can reintroduce fences - and I'd start with grid work of little (18" or so) fences so that he can also focus on building his hind end and topline. Cavaletti will also be a huge help and you can vary the distances to vary the length of his trot stride to work different muscle groups.