I agree, ground work should be started asap. Young bones shouldn't be taxed in a round pen or on a lunge much though, so I keep foals on a lead line and just teach good manners in general, with only short lessons in the round pen every now and then.
From yearling until 3, I like to take the horse on trail walks in hand, no other horses, and occassionally ponied from an older quiet horse. Nothing tough though, just enough so the horse gets to see the world and learns how to behave in new and "scary" situation.
At 2, I like to get the horse used to a saddle and bridle, advance the round pen and lunge work some, and introduce ground driving.
At 2.5 to 3 years old we work on long lining and learning to stand for mounting and dismounting (with maybe a few steps of walking, but that's it).
Between 3 and 4 years old we work on the basics of riding, WTC, stop, turn, backup, and some trail riding. Of course, this will go easily because we've already done all of it on the long lines and with ground driving in the arena.
At 4 I will start harder work and longer trails. I won't jump a horse or work on roll backs, spins, slides, etc. until he's 4.5-5yrs old.
I use this schedule as a guide, as all horses are different in their mental maturity and what they can handle. BUT, ALL horses' bones mature about the same. Knees close at 2-2.5 yrs old, hocks close at 4-4.5 yrs old, and all the bones are done growing at 5-5.5 yrs old. Big geldings (that will mature over 16h) take an extra 6 months for knees, hocks, and full bone maturity between 5.5-6 yrs old. This is a scientific fact
, studied by more than a few equine scientists/vets in the field and in the lab.
Starting horses too hard, too young is why we see 7-8 yr old horses needing hock injections, young horses with bone chips, horses developing arthritis at 13-15 yrs old, etc. A horse SHOULD be riding sound, barring any illness, injury, or genetic defect, until nearly 30 yrs old. A horse SHOULD be sound enough for jumping, cutting, reining, etc. ("hard work") until 18-22 yrs old.
If you're a light weight rider and KNOW how to properly condition your horse for work, then yes, you can start them early (at 2) and they will be sound for life. But, there aren't many people that REALLY know about horse physiology and strength conditioning starting young horses. If there were, we'd see far fewer soundness problems.
Greed and impatience cause people to push young horses too hard and fast. 3yr old Futurities (cutting, reining, WP, etc) are partially to blame. Who wouldn't want to win some money? The racing industry starting horses at 18 months old is also partially to blame ("the race people do it, why can't I?"). But the biggest problem is just plain ignorance... I see it on horsetrader.com and equine.com all the time. Some dumb a** with their kids riding a yearling, or their teenagers running barrels on a 2yr old, or a 2yr old draft pulling a big heavy waggon full of people, or their big husband/brother/dad/uncle galloping a 2yr old down a hill, etc. People don't take the time to think about the age or development of their horse, or learn about it. The internet is full of information if you just take the time to look for it.
Two very good articles on this topic, both written by vets. Every horse person should read them. Esi Knowledge Base - True Collection And They Call Us Horse Lovers - Articles