^^Exactly. All you can do is make sure that they are working correctly (and enough, 10 minutes of walking in a flat arena once a week doesn't qualify as being "worked"). Also, make sure that they are getting good quality feed for the amount of work they are doing. A horse that's being worked hard 4-5 days a week and being fed nothing but grass hay will not be taking in enough nutrients/fat/protein to build muscle. On the other hand, a horse that is being ridden once a week for 15-20 minutes won't benefit from a bunch of high fat, high protein food. That will just cause them to get fat, along with creating the risk of other health issues.
Beyond that, it is all up to what genes mother nature gave them.