Trying to not make this into a controversial topic either. I see both sides, without prejudice. I sometimes wear my helmet, I sometimes don't...depending on what we are doing and my comfort level for that day doing it. Maybe, for me it is a mental thing. Because when I look back, everytime I've fallen, my head never hit the ground, but my back and butt did. I have had a broken finger from holding the reins when I fell and plates and screws in my ankle from getting off of a horse. However, I was on a trail ride, a place I always wear a helmet, and I was riding my percheron through a canopy of low trees. My focus was on the ground, which was going downhill briefly and then going right back up hill. I was wondering if my horse would jump or simply walk.....well, I did not see the 8" diameter tree limb that hit me right above the brim of my helmet. My brim prevented me from seeing above, as I was looking down. The person behind me said I looked like a pez candy dispenser when my neck and body got knocked backwards. If I hadn't been wearing my helmet, I probably would have been knocked unconcious for sure or worse. I had whiplash for a few days afterward. With that being said, when I ride around the ranch or in the arena I usually never wear a helmet. We did mounted yoga in the arena bareback and I was even wearing flipflops.
I will also say, I work in the operating room. I have seen many horse accident injuries and motorcycle injuries...and they weren't head injuries....some had helmets, some didn't. There is no full protection from injury and the injury depends on what happens in the first place. It's one of the risks you take for the love of the sport. Even the jousters hurt themselves with full body armor.