Oh, I see; you consider horses pets. I do not. To me (and the local authorities) they're livestock.
What you need to do is find out what your municipality requires concerning the housing of livestock in your area. You'll need to know the minimum acreage requirements, fencing requirements, as well as what the local laws are concerning shelter.
You also need to find out if your area is zoned for livestock. If it's not, then you won't be able to put a horse on your land regardless of what the neighbors do or don't have.
Horses are a continuous expense. Their purchase price is actually the smallest amount you'll ever pay when it comes to them.
Horses also live a very long time. They're not like dogs or cats whose lifespans are measured in 10 to 15 years, they're measured in decades. I have a 24 y/o pasture pouf who was retired at 21 due to arthritis issues. You have to remember that even if a horse isn't being ridden, it still eats and needs the same care as one who is. Sometimes it needs even more care, depending on its physical issues.
So you're asking how much an animal costs is immaterial. The costs vary wildly depending on the breed and type you choose, as well as whether or not they require special shoes, or ongoing medical care.
That is why I recommended you board for several years prior to bringing the animal home. If the horse is in an environment where someone actually knows what they're doing, the chances of you farking it up become less.