I'd say at least half of the adults at my barn don't wear a helmet, including my instructor (I've seen her wear one just a few times, once when riding my horse... hmmm, maybe I should take that as a sign, lol), but I always do. The one and only time I fell off a horse in my 15 years of riding, I hit my head (helmeted) and am sure I would have been injured badly if I hadn't been wearing it. Actually, I usually wear a helmet when working with horses on the ground, too. While the risk is not higher than while riding, it is still significant. A woman at my barn was walking her horse when he slipped and fell, pulling her down and hitting her head. She was injured badly, but her doctor said her helmet saved her life. I wouldn't tell other adults what to do, but people are often bad at gauging risk and so information can help people make more informed decisions. Yes, riding in an English saddle increases the risk of injury about 70%, but I still choose to ride in an English saddle with that knowledge. If people know the risks of not wearing a helmet and choose to do so anyway, then at least they are making an informed choice.
I feel a bit differently about children, though, as our role as adults is to protect them because they are not able to make those informed choices because of the lack of a fully mature brain and abstract reasoning. Also, nobody has mentioned this yet, but children are actually more vulnerable to head injuries when they ride than adults, due to the larger size of their heads relative to their body size.
Now, as far as the statistics go, when injured in a fall where they hit their head, only 5% of people wearing helmets suffered a brain injury compared to 60% of those not wearing a helmet. That is a 12-fold increased risk of brain injury, or 1200%. Yes, there are many factors that go into safety around horses, but it seems to me like this is a huge one, especially for kids.