I don't see as a problem, tho I would not let a small child on the ground on their own. A little one that young should be in the arms of a grown adult where you, as a responsible adult can move out of the way in the case something were to arise. A child will not be able to move quickly enough and you're going to be asking for a disaster.
I had my son around my horses from as soon as the pediatrician cleared him to be allowed in open air. (My son was a preemie and had to be on oxygen after birth). I was searching around but couldn't find it, I have a picture of him riding double with my on my old (extremely safe) belgian cross mare at 3 months old. He was riding his own pony almost before he could walk (see picture below). Horses have been a way of life for him, he has been brought up around them and all other animals with particular emphasis on learning how to act appropriately around them, and how to be safe.
I am as over-protective a mom as they come in some situations. My neighbors let their kids run the woods behind their house unsupervised and I think that's certifiably insane. (there's a lake back there as well). I never let my child even in the yard unattended, not to mention off my property. Yes he is only 6, but several of their children are even younger.
This is Spencer on his first pony (mini) Flaxie at about 11 months old:
However, I put my son on horses, and bikes, from a very young age. I've let him have his scrapes and tumbles, and let him get himself filthy. He's a kid, and oversheltering in these circumstances is a dis-service, in my mind. He's fallen off my mare Freyja (when she was standing still, no less, he thought it was funny to wiggle around and down he came.) He's had countless skinned knees, he's drank out of the hose, he's helped clean stalls, and he's fallen face first into horse poo. He's all boy.
And for the person who asked about riding during pregnancy - my Dr actually recommended I continue riding during my pregnancy, but forbid me from riding strange horses, jumping, or riding hard. He said it was good for the pelvis and and it was exercise when a pregnant womans' exercise options are limited. He even wrote me a note to allow me to continue work as a trail guide at the guided ride facility I worked for, and allowed me to fly from Indiana to California and back in my 6th month. He had every reason to, my pregnancy up until near the end was picture perfect and had no problems at all, I gained minimal weight (15 lbs at the end of pregnancy), never suffered for lack of energy and never had a single bout of morning sickness.
However, I DID have to stop riding at the end of my six month, I got put on full time bed-rest because I had gone into pre-term labor. Let me stress this had nothing to do with horse riding, the premature labor had more to do with other issues and the Dr was confident riding during pregnancy had been a good choice, and was even to some extent instrumental (strong, well developed pelvic floor muscles from riding with the extra weight) in keeping little mr. Big hurry from making his appearance in the world for another month...in total keeping him in until the end of my 7th month.