I hate to beat a dead horse...but in the first set of pictures too I had to recheck your post to see his age, I would have guessed in his 20's because of the lack of muscle tone, and it's not just flab like on unworked horses, but he's hollow through the hips and shoulders the way older horses age and shed muscle. I am not doubting his age, just saying he's really got a ways to go with muscling. In the photo from teh winter he looks younger, so i'm also wondering if it's his coat and just a not flattering picture in the first post. On the plus sign, you can see his skeletal structure and it looks good. A little straight through the shoulder, but I like his neck and hock angle. Nice legs, good bone.
I would second the advice and put him on a really good feeding program with muscle building supplements added in. Good luck!