One of the things that bother me is that they bought a new horse and new gear but wanted you, for free, to work with him. And then they don't work with him themselves???? Sometimes I don't get people....But I know the type you are talking about.
If he's 4 and 17 hands, he's young for that breed....he's going to get heavier. On the other hand, it would be sad to see such a beautiful horse sit in a pasture with bad feet....
I have a belgium on a farm with 14 other horses. He is by far the friendliest horse. When we have kids out to ride, we lead him with the kids on. When kids are around, he mellows out and never moves his feet around - he stands so still. He wasn't worked with until he was about 4-5 either. But once he got individual attention (basically babied), he came around. (Treats helped too)
It is ultimately your choice. I'm a sucker for the big guys but I'm prejudice now
Do what you are comfortable with.