I personally don't give a care in the world what people choose to wear that is their decision... but I personally refuse a helmet, and have for the last thirteen years of my riding/instructing/trailing/occasional jumping for fun.
I have one opinion about helmets, that is too many people use them as a safety blanket.
If you can't ride to save your life then wearing a helmet isn't going to help the other 80% of your body when you eat dirt. Also if you ride correctly, at least this is the case for western riders, you shouldn't ever fall off head first. Unless by some crazy random freak accident. As a former instructor I have seen a lot of falls but no one I have ever seen, has actually damaged their head.
I feel like if helmets are so important to our well being on a horse, then by golly we must wear them everywhere just to be sure we won't be caught off guard.