Not at all! I'm also a first time horse owner and I got my mare when she was 23. She's now 25, almost 26, and you would never know it.
I think it really depends on the horse. Like anything, some age a lot faster and worse than others.
I have no doubt that, barring any unforseen events, I will still be riding my mare when she's 30.
Of course, it does take them longer to get into shape so I try to keep my mare in pretty steady medium work all the time so that she stays in shape (and so that she doesn't kill me with her energy, haha).
The other concern I've had is when I'm riding with people with younger horses, they tend to just want to have fun while I'm more concerned about the condition Lacey is in and how she's holding up. I'm a lot more cautious with her than I think I would be with a younger horse because at her age, she's more physically fragile. I also have to be very aware of her fitness level and when she seems to be getting tired since she will go ALL day if I let her, but her body isn't up to that.
Other than that stuff, I really feel good about owning an older horse. I like thinking that I know I'll own her for the rest of her days and I like making her later years her best. I also really like finding things that we both like to do and just doing them, without any pressure to be great at it or anything. And, I love that she's been around long enough to see pretty much everything so I can take her anywhere and never really have to worry about her getting spooked or out of control.
Older horses are really great. They do involve a little more care but I think they adore you more for it.
Fabio - 13 year old Arabian/Lipizzaner gelding
Hazel - 14 year old Angora goat
Atticus - 4 year old LaMancha/Alpine cross goat
Rest peacefully, Lacey.