Like a great many things with horses, the answer is "It depends."
But let me put some framework around that and make it a more useful answer.
MOST children are too young to start formal lessons before the age of 7. In order to get anything out of a formal lesson, a child has to have body awareness (If you tell them to move their lower leg, they can identify, isolate and move just their lower leg) a pretty good attention span and the ability to follow directions. 7, in general, is an age where those things come together, some children will be ready at 6, others not until 8 or even 10. In addition, before you start formal lessons, you want to know that they can graduate to independent control over the horse within 3 - 6 months. If you keep a child on a lunge line or lead line for too long, it's much harder for them to become independent riders later.
Children that start younger than 7 without the body awareness and attention span have to be "babysat" and monitored constantly. My own daughter rode in front of me on the saddle and was leadlined from my horse practically from birth; and I gave her casual instruction and "mini-lessons" all along; but she still wasn't ready for "real" lessons until 7.
If you have a friend or neighbor whose child is taking lessons at 4 or 5, don't panic or assume it's unsafe. It can be safe, with the right horse and supervision. Whether or not it's a good use of time and money is another question entirely.
Oh, and welcome to the forum!