Both 'sweet feed' and grain are generally pretty bad for horses, sorry to say. To save repeating myself too much, look up the thread 'vegetable oil or corn oil...' to learn a bit about what's good, bad & why for horses.
Unless there is a lot of poor quality long stemmed grass, I'd want to see them getting hay too. Horses need at least 2% of their bodyweight daily in fibrous forage. If he's underweight, or a good weight but losing, that would be my first answer - give him free choice - or at least plenty of hay.
Also the question of how often he is fed will effect how he's able to use it, and if being fed grain in only 1-2 feeds daily, that may be *contributing* to his weightloss rather than adding anything to him. Another question is nutrition - if he's not being correctly supped for balanced nutrition, it's possible it's a deficiency or imbalance, rather than lack of calories. ...And as you mention the word 'plump' and many people seem to like looking at mud-fat horses, I have to ask - is it possible he doesn't need any more weight now anyway because he was obese before??