^ That has nothing to do with whether you wear a helmet or not. I wear a helmet 99% of the time but I also have been known to ride barefoot, in shorts, in a halter (Or even just a neck rope).
I think it depends a lot on how you were brought up/where you ride/what discipline you ride. I started out in pony club where helmets are compulsory. Every discipline I ride in except one helmets are compulsory. One of them is campdrafting which is as close to a macho, male dominated, western type discipline you can get outside western - And yup, all those men who train horses for a living sans helmet don one for the competitions, every weekend. I don't hear them complain or call each other sissies, either.
Growing up where helmets are a commonplace thing, I admit - I think people who don't wear one are a bit dumb. I think people who don't wear one when riding on/near roads are downright stupid.
Doesn't mean I don't like them as a person, or that I would ever say anything. But I still think it.