I was just going to echo the feed comment.
I used to have my filly (shown in the picture) at a place where I wasn't told how much she was fed. Then one day I helped with chores in the morning, afternoon, and night. You think the "fed 3 meals a day" thing sounds great until you see how little they're getting. My filly received a flake in the morning (which varied by the person giving it, but was usually about 2 inches thick, or 2-3 lbs), half a bale in the afternoon (that she fought over with 4-5 other young horses), and another 2-inch thick flake at night. I calculated out what she was probably getting per day, and it was probably about 10 lbs. She was hard to handle, did not listen well, and was just naughty in general. I switched barns, upped her feed to 18 lbs of good alfalfa a day, and lo! After 2-3 weeks, her attitude and manners were better than that of some of the 20-year-old horses at our barn.
Personally, I'd rather have a fat horse that receives all the needed nutrients at the time they're actually needed, than a lean horse that ends up having health problems later on in life. I'm sure you know how that goes! We gotta make sure our babies have adequate nutrition early on, or we'll have all sorts of crappy problems later on down the line.
Yours sounds like a trooper that's willing to fight for his food. I was told that my filly kicked the crap out of the other horses, whilst trying to hog the hay. She still didn't get the amount she really needed. If it makes you feel any better, you could always ask them to delegate a specific portion, like 8 lbs, to his stall. That way you know he's at least getting that much per day. Knowing that my horse isn't hungry ALL the TIME gives me peace of mind!