However fitness is fitness, and it is very important to cross train!!!!!
I know of two dressage riders this year alone that have pulled muscles in their thigh/hip area. It is very important to strengthen things that are not worked while riding so that during other exercise (ie running - as both these riders found out) there is no over stressing of weaker muscles. You need to strengthen the body as a whole not just think of what will strengthen riding muscles.
It's also important to find things you like to do. I am A-type personality so I like going to the gym and lifting stuff and going fast on a bike. I always plunk myself down beside the fittest, hardest person in the gym and go as hard as them and compete with them. I'm going to try x-country skiing this year as a "relaxing" activity haha so we'll see how that goes because I can't bike in 2 feet of snow!!