Ponies are good IF you can find a good one. Well broke, good mannered ponies are far and few between and when you can find them, they are worth their weight in gold.
We skipped the ponies (I'm so picky, I'd have never found the right one) and my daughter rides the horses. She is 6 and doesn't quite weigh 50 lbs. The 3 she rides the most are my 25 yr old retired reining mare that's just a hair under 15 hands, my 23 yr old retired WP mare that's 15.2 (she shows her in small fry pleasure) and then at the other end of the spectrum (and her favorite to ride) is my 20 year old 16.3 Hanoverian mare.
I wouldn't count out a horse because of size, with kiddos, having a gentle disposition and being dead broke are the most important qualities. Good luck on your search!
Life is like a camera. Focus on what's important, Capture the good times, Develop from the negatives and if things don't work out, Take another shot.