I'm going to start out by saying I'm by no means a professional horse trainer - I just start colts for my own use, and will sometimes sell them down the line, once they're well-broke. So, I guess, take my advice with a grain of salt (but you probably already do that with most people here).
That being said, I have a young mare that sounds like she has a similar temperament to the horse you're working on. I'd done lots of ground work with her before hopping on, and she was pretty good with everything. Round pen work, desensitizing with plastic bags etc..., ground driving - we spent quite a bit of time on this kind of stuff. Like your horse, she wasn't handled much before I got her.
But, once I actually got on, she was extremely tense for a long time. She caught me off guard once, crow-hopping, and I did bail (I got right back on, though, and the rest of the ride went okay. My pride was the only thing that was hurt). She was constantly thinking about throwing me for at least the first 20 rides, maybe more. Her back was humped up constantly, and she'd pin her ears whenever I asked her to yield or circle to the right, even though I have good hands (she really was left-handed, and still is a little).
Just getting lots of good rides on her seemed to help. I kept the rides pretty short at first because I wanted to keep things positive. She's a really sensitive horse, and I found that switching my cinch helped, so I'm always really conscious about my tack with her, even more so than other horses.
Once I started to relax a little and have fun, she seemed to loosen up, too. This is easier said than done, I suppose, but being tense doesn't help, especially with a touchy horse. Also, I found that she really likes to move, and once I was more comfortable with her, we did lots of trotting. She likes to stop hard, too. So I think if you can make training sessions positive for your colt, that will help.
I'm happy with the way my mare is turning out now. She's often really eager to go for a ride, which wasn't the case in the beginning. She seems like she's going to be a very reliable trail horse, and she's real cowy. She doesn't spook at anything, she's lively and she's very responsive.
So being patient and listening to your horse really pays off.
But that being said, I don't think she'd get along with every rider. I think that anyone who wants to ride a young horse needs to be an excellent rider and relatively fit, no matter how level headed the horse seems. You can't control what he ultimately does with the horse, but hopefully he'll take your advice (whatever it ends up being).
Good luck. I hope that's somewhat helpful.