I am of the same mind as Kevin. I don't wear a helmet but I would never fault others who do. OP, you should wear whichever you want to wear. If you feel the need to wear a helmet, then do and don't concern yourself with what others say about you. If you want to wear a cowboy hat, then do and don't concern yourself with what others say about you.
There will always be people to stand up and scream "You're wrong!!" no matter what choice you make, so make the choice for your own
reasons, not anyone else's.