What is the weight of the hay you are giving - "flakes" is not a good measurement when it comes to amount of hay because it can vary SO widely. For a horse that is having a weight issue, it is important to know exacty what you are giving them to identify where changes should be made. You're goal for increasing weight needs to be 2-3% of the horse's goal body weight in hay as a base (free choice is best, if at all possible). Also, what kind of hay is it that you are feeding? Increasing the amount of hay will not do a lot of good if it is hay that is not going to be useful to his body.
What sort of "grass" pellets are you feeding? Also, again, weight, not "scoops" needs to be determined in the grain(s) you are feeding. Take a "scoop" of each of the feeds you are using, put it in a baggie and go weigh it (grocery store scale, feed stores, hardware stores that sell bulk nuts/bolts, etc are all good places to find scales).
Until you know what you are feeding, there is no way to advise as to what to change. What is the amount of work the horse is doing? When was the horse last vetted and de-wormed?