If the horse is under 1 year old I like them to live life as babies. The weanling should be easy to handle, easy to catch, friendly, accept being touched anywhere, stand tied patiently for a short time, lead well, be used to a routine grooming, accept de-worming, pick up all four feet and be learning to have good manners with the farrier. They should know that humans are not chew toys and not be allowed to develop bad manners but receive plenty of hugs & kisses :) This is all easy to accomplish and does not require "daily work" by any means. I go weeks in between haltering my foals and when I do go get them they are happy to come in, stand, be groomed, tolerate a farrier etc.
I feel that a yearling's brain is a little better equipped to have a job but still, its one more year of baby time for our yearling's. Living in a herd environment is very important to the youngsters! The first two years of life are beneficial to spend learning from their elders rather than having a human formulate an agenda for them. During the yearling year I expand what I wanted from the horse as a weanling. Stand tied longer, for example. Loading into trailers, walking through puddles, stepping over logs or ground poles, clipping, bathing, fly spray, pony down the trail, lead away from the herd, wear a blanket, halter class and show experience, accept cold hosing on legs for that day they injure themselves, stand with your foot in a bucket of water for the day they need to do so due to an abscess and anything else that you can think of that doesn't require "work". I have typically done half of this once or twice with my horses as weanlings, I just make sure that they know it as yearlings.
Once the horse is two years old I would introduce the bridle (after his/her first dental visit), ground driving and light lounging. I do not do much on the end of a lounge line with a mature horse anyway but I feel that it is especially best for young joints to wait until a horse is into their two year old year. A horse will learn to walk, jog, trot and canter in a circle just as easily as a late two year old as he/she would have any earlier. It is so, so, so much better in the long run health wise to give your horse that extra year.
I do not get on a horse reguardless of its breed or size until the horse is a full three years old. Any sooner and you are pushing your luck IMO. So age three is when I begin workouts, wearing a saddle and undersaddle training. Three years old is considered their light work year (limited stress on the joints, no tight circles, shorter workouts than a mature horse) while they are green broke and still maturing.