I think he looks thin. Not call animal control thin, not malnourished & you're abusing him thin, but definitely thin. You can see his ribs, and his neck looks thinner than I think it should look. You asked, I answered. ;)
Yes, TBs are harder to put & keep weight on, but I believe he needs more weight.
Before you start spending more money on feed, have you had his teeth checked and has he been dewormed? Could there be any other medical issues keeping the weight off? If those have all been explored, then move on to feed issues.
Definitely ask the vet what he thinks of his weight & body condition, and ask for his recommendations on upping his weight. Many people swear by BOSS (black oil sunflower seeds) as a weight enhancer. Soaked beet pulp is another good weight builder. Even plain old oil (vegetable, sunflower, etc.) poured over his feed can add calories. Of course, there are lots of commercial mixes & feeds that are good. Also, you didn't mention what kind of hay you're feeding. Perhaps adding alfalfa will give him the extra calories. Take a look at your pasture as well. Just because he has access to a pasture doesn't necessarily mean it's all nutritious, or the highest level of nutrition. You might need to offer hay even while he's on pasture.
Good luck. I hope you're able to find a solution for him!