I don't wear a helmet very often. But I required my daughters to wear one every ride. Mainly because they were not as experienced of riders, didn't always have as much control of the horses as more experienced riders have. They also don't have the strength to "man handle" a horse that is mis-behaving. So I believed they had a "Greater Risk" of possible being thrown than I do.
I do wear a helmet when I work young horses. I do wear a helment when I'm in evironments where the horses are more likely to get spooked or excited. i.e. Riding in an endurance race where you have 100 strange horses all moving at trots and faster speeds. I just think the risk increases because of the everything going on around you.
Riders need to access the risk. Their skills, their horses, the enviroment that they are riding in and make the decision about wearing safety gear. Many of us grew up in a time when we never wore helmets for any activity. I can't tell you how many hours I've spent riding bikes, skiing in addition to riding the horses with out a helmet. As personal protection has become more recognized as being valuable, more comfortable to wear and more cost effective to buy, we see helmets used in more and more sports. People almost never ride a bike now days with out a helmet, Half the people on the mountain skiing now wear helmets.
There is nothing wrong with wearing a helmet and I would never make any kind of commet to discourage an individual from chosing to wear one.