Well I'm going to be the odd duck here and say sometimes. When I was little I wore one every time (and still make the kids who come out wear one) but when I grew a bit older I started to only wear on when I felt I had the highest chance of getting bucked off and trampled. Like when I ride my mare (and I'm not one to say this but I literally trust her with my life) I usually don't wear a helmet, if I'm riding her english that's a different story, as is bareback cantering. If I'm riding a training or spooky horse I wear a helmet. The horses that I don't wear a helmet on are ones that if they do spook it's mild and I've never had a problem sitting it or correcting it before it happens. If I don't know the horse like the back of my hand on my helmet goes.
ETA: I also see it that a helmet will do very little to protect your head if you fall head first (I witnessed this with a kiddo and it was frightening to say the least. Her helmet shattered and she had bits in her head.), or snap your neck.
That being said please don't berate me for not wearing one 100% of the time. It's my body and my potential mistakes. We're all entitled to our opinions and such.