sounds like you just know they are going to over do it and not really as upset with this one incident. Poor horse. I do think some horses can abe started this young. So much depends on the horse, the breed, the build, etc. At our barn there is a 4 yr Oldenburg and she has only done ground work, she says a trainer told her to wait to get on his back until is 5!
He hasn't really filled out yet and is all legs, but I think 5 is pretty long to wait.
By 5 most horses are ready to do whatever job they are bred for, but I guess some breeds are different.