For me, it depends more on the horse and less on the age.
I was given my mare when she was 23 (I was told she was 21, of course) and now, nearly 5 years later, we're starting to hit some VERY mild "old age expenses". She doesn't require a joint supplement, she's an easy keeper, she's perfectly sound, basically like a young horse in the body of an old horse. :) She's only now starting to have a few very mild tooth issues that aren't causing any problems, just require watching. But that is it. She has other health issues that are not age-related and are expensive...but they add to her charm.
We do/did (she just had 6 months off for a suspensory issue but we've recently been ok'd to start riding again so we're starting to build fitness and muscle back) ride some pretty intense trails very extensively - very steep, etc, and she does fine. When she was really fit, she gave no thought to going out for a 3 hour w/t/c/g ride over really varying terrain. If anything, she loved it!
Of course, I do pay special attention to how well she's cooling out and I don't expect her to trot/canter/gallop for longer than maybe 3 minutes each set before giving her a short walking break (she could handle longer sets).
The one caveat in your, the OP's, situation that might be an issue for an older horse is the issue of weight. My mare does great but I weigh about 145lbs and that's pretty much the top of her "comfortable limit." Any heavier and it's really obviously too heavy for her to be comfortable. Depending on your weight, a younger horse might be a better option. Of course, some older horses still have great backs (my mare's back is her biggest flaw - it's really long and "weakly built") and that could be something to add to your "wanted" list.
Personally, I really love owning an older horse, I know that one day I'll be her retirement home and that's my favorite. Riding her takes on an extra bit of happiness because I know that her riding career is pretty finite in terms of a horse's life ling riding career.
Personally, I think an older horse is a GREAT first time choice, but, that being said, a younger well trained/very broke horse could also be perfectly acceptable. I would focus on looking for well broke horses and pick the one you have a connection with - old or not, riding is more enjoyable with a best friend!! :)
Fabio - 13 year old Arabian/Lipizzan gelding
Hazel - 14 year old Angora goat
Atticus - 4 year old LaMancha/Alpine cross goat
Rest peacefully, Lacey.