Personally, I think it's really, really stupid to ride without a helmet. You might be really secure in the saddle, your horse might be bombproof, if you do fall off you could have any number of accidents that the hat won't protect you from - but horses are unpredictable, and riding is dangerous. I think that you should always wear a helmet when riding a horse.
I don't know if this is a UK thing or not, but I don't think I've ever seen people riding without a helmet! At every stables I've ever seen, everyone wears a helmet. It's not a case of 'barn rules' - it's more of a mindset that if you don't wear a helmet, you don't ride.
I would think pretty poorly of someone I saw at my stables riding without a helmet, even if they were one of the really good riders there, because they'd be setting a really bad example. I saw someone getting on her horse with a friend holding the horse and her helmet - and another girl walking past told her off for not putting the helmet on before she mounted.
I just think that it's completely not worth the risk. Okay, so your helmet might cramp your hairstyle or you might not like the feeling of wearing one, but it could also save your life. And I don't know about the rest of you, but within five minutes of getting on a horse I've completely stopped thinking about things like what's on my head!
As one of my friends said - she's had bad falls, and fell on her head once. She can't remember anything for five minutes on either side of the accident. But that helmet changed brain damage into a mild concussion.