Yeah. It sure depends on the horse and can go either way.
A roper friend of mine had an old, steady horse about 18 years old. Some neighbors, non-horsey, stopped at his arena to watch him and friends practice. Their adorable little blond girl had horse shaped stars in her eyes.
Nice neighbor offered to let her ride his old gelding. Lots of older kids had learned on him. He said "He's as gentle as a lamb." Little girl gets up, is walking along, just a passenger, nice guy is walking next to them, and the horse suddenly crow hops and runs off, leaving her in the dust, unhurt.
Girl was okay. Parents weren't freaking. Nice guy gathers up his horse and I hear him say, "Baaaa, you sonuva_______!"
On the other hand, I always had some projects around. At least at first, when the kids would come to the corrals to check them out, most of the horses would stand perfectly still and let the kids handle them, when they still had the wiggles with me.