I don't usually, because it's habit and I don't even have one right now but plan to get one. I keep thinking I should get in the habit, but never have since I was a kid. When I learned to ride only hunters/jumpers wore helmets, or you wore one when you showed, and I didn't do either so I think that's where the habit came from.
Now when I wear one, I feel really confined, claustrophobic and HOT ha ha. And yes, I've been tossed through a wood rail fence and ended up in the hospital and yet it still doesn't make me wear one. I eventually want to try Cinny over fences though, so I'd better buy one and get used to it lol.