Certainly not! As long as he's healthy and sound, you can do distance. Make sure you condition smart; don't rush or push too hard. That's when horses get hurt. Find yourself a riding buddy and go get some trail miles, since he isn't seasoned.
A friend of mine started her newest CTR horse at 13. The horse had been shown in halter and hadn't been ridden in eight years. She took a couple weeks doing long, slow walk/trot exercise. After three months, she did a little two day 35 mile CTR and got best conditioned.
The horse could go further with more work, no problem.