I've rode both TBs and QHs, and honestly with the right horse I can't see much a difference. The TBs I've ridden have been rather old, 2 of them over 20. But the still seemed pretty chill for a normal stereotype of the hot headed, just wanna run TBs. They weren't off the track as far as I know. One was an older dressage horse, who just stood in the middle of the arena obediently and practically fell asleep while I had my lesson on my POA gelding. There were horses moving around but that old guy never took a step. Next was my friend's horse. She was a gentle giant, or at least seemed really giant to me then. She stood there as kids crawled all around her, and was used as a trail horse. Then that horse's daughter, who was only green broke. She let my friend crawl all over her, and even ride her. She's only been saddled a few times but has been ridden many times. But they were pretty good horses, and with the right personality any breed can be a great horse. Some have a more common personality then others, but you can still find one great horse.
I've ridden a buttload of QHs, and would prefer them any day simply because I have ridden them so often. My friend has a quarter horse that I rode who was an excellent barrel racer and is really good at eating. I've actually fallen off more quarter horses than thoroughbreds, but that's because I only fell of a QH once because he spooked. Out of the 3 or 4 thoroughbreds I've ridden I've never fallen off, but that's sort of a dumb statistic because I've ridden alot more quarters.
Out of every breed you can find some good, some bad, and some ugly. But either way I don't think you'll have to much problems transitioning to a TB. I've heard alot of OTTBs being pretty good horses. Especially since this guy is a bit slow it might make it better for a trail or pleasure riding horse. Most OTTBs just need more experience in learning how to have more than one speed, but since he hasn't raced all that much maybe he won't be stressed with learning something new.
I think your horse will do just fine, just work with him a bit to give him a refresher on easy riding and you should be good to go. I think every horse has alot of potential to do just about everything.