As a general guide, 5 years old but White Foot is right - it depends on the individual horse. It also depends on the level of training the horse has had on the flat. Personally I don't like to jump a horse until it is consistently scoring above 60% in level 1 dressage (also as a guide).
I would say start some pole work around 3 and a half just trot poles and stuff then maybe VERY small cross rails at 4 and only a few times . At 5 I would start jumpiong anything higher then a loow cross rail.
I don't jump any horse before the age of six, at the youngest. I'll do trot poles with 3 and 4 years olds, but nothing more strenuous than that.
Horses mature from the bottom up, so chances of you hurting their legs are much less than the risk of hurting their backs, because the back and base of the neck are the last to fully develop. Also, keep in mind that the growth plates in the legs are perpendicular to the weight they're carrying, while the plates in the back are parallel to the stress, which makes them less resilient and more prone to damage. Those bones aren't fused until around 5 1/2 years old, regardless of the horse's gender or breed. In my opinion, it's best to play it safe and stick with light work until the horse is completely done maturing.
Wait until the growth plates stop changing, and then you can start jumping them safely. Most breeds' growth plates secure around 3 or 4, but some breeds, like arabs, mature physically and mentally slower and shouldn't be jumped until much later.
I jump my 5-yr old TB very occasionally (once a month or so) and nothing more than 2'. He's my future eventer, so I want him to get used to jumps and learn to use his body productively over jumps, but I don't want to push him into strenuous mental or physical overload by jumping him more often than that right now, especially since next spring and summer we're going to really start cracking down on the eventing lessons.