You didn't mention your goals in riding. If you want a specialized sport, then buying is a big commitment. If you want trail riding, entry-level dressage, little jumps, non-competitive games...then buying isn't a lifetime commitment. Where I live, you could buy an OK horse for under $1000. Nothing special. But OK.
And if it works, fine. Spend time improving the horse. Teach it basic barrels, or entry dressage, or to go on long trail rides without getting spooked. If you are consistent and fair, most under $1000 horses will get better from being handled by someone who provides certainty.
But if it doesn't work out, you can then give the horse away to a good home and lose a maximum of $1000. Our little mustang was free. The woman bought him for her husband (200+ lb husband and 13 hand mustang - am I the only one who thinks that is strange?), but he turned out to be allergic. They had one child, and wanted to have more and decided they didn't have time, so they put their two horses up for sale. The larger one sold, although some folks told me the larger horse was a PITA. The little horse didn't, so they offered him for free.
Cowboy now lives here. He's ridden about 1-2 times/month. Good little trail horse. Still gets nervous about arenas. I may start riding him (175 lbs seems a bit much, but he's stocky and I don't think 2-3 times/week for an hour each will be too much).
It isn't a crime to sell a horse. Some horses will match your personality, some won't. You often won't know until you've had the horse for 6 months. Sometimes the work of training their bad habits out builds a bond. But either way, it isn't a crime to sell the horse if your life changes. The husband in me doesn't like 20+ year commitments I can't back out of. But it isn't a 20+ year commitment.
When you own a horse and ride it regularly for a few years, you get to appreciate their minds and personalities. You find out what they are really like, and they learn about you. That can be good or bad. But if you work them right, they will get better and gain or at least hold value. And if you don't buy a $10,000 horse, you can't lose $10,000. If your life changes 4 years from now, like life often does, sell the horse.
NOTE: The husband of 25 years in me noticed a while back that my wife is less annoyed when I buy a $600 revolver I don't need if I don't get annoyed when she buys a $800 piece of furniture we have no place for.