I think it has to do with how public accidents are. We don't hear of many horse-related accidents on the news, but a climbing accident is often sensationalized. Perhaps this is because when a climber or a mountaineer gets into a bad situation, public service authorities and resources often have to be brought in to help them. The remoteness of these accidents also require extensive equipment (i.e. helicopters, rescue teams, etc) that make these rescues much higher profile and more newsworthy. Most horse accidents occur in easy to get to places serviced by a standard ambulance so it is not such a big deal.
I also ride motorcycles, a sport that is perceived to be very high risk. Statistically speaking, I believe riding horses is riskier, however, this is not the perception. Again, I think it has to do with public image. Motorcycles travel on public roadways, so when there is an accident, it is often out in plain view for everyone to see. In addition, when there is an accident, the carnage (pieces of bike parts flying across the road) is striking. Horse accidents are not. Injuries and damage are more frequently internal and not as visible. Horse accidents are also more frequently limited to horse-related venues and witnessed more by horsey people, not the general public.
There are also differences in the likelihood of death in different sports. The risk of injury on a horse is much greater, but the risk of death may be less than with mountain climbing or motorcycling.