There are always people that will come to you for lessons.
If you have your own land, you can get by with something to block wind, in warm climes, or go all the way up to a couple of stalls, tack and feed room. How fancy depends on you. Many lumber yards build run in sheds, and deliver them. 1500 is about what I figure, if you have to have one built. I've had one done for 800, but was 8x12 and no interior plywood on it.
If land is not fenced, then will have to do that.
And if you are going to have to feed and hay daily, then one horse will go through a bag of feed each week by itself. Cost will depend on how good a feed you use.
Hay takes about 15 small squares a month, at various prices.
If you have pasture, then you will not have to feed more than likely at least 6 months of year at minimum.
Shoeing/trimming 40 or so for trim, 60 and up for fronts only. Again, depends on area.
Shots 50 or so each horse per year if given by vet. De-wormer 5-20 dollars each time, and if have vet do fecal, factor in vet visit.
Figure to spend at least 1500 to get started with barn, any fencing needed will factor in more, feeder or troughs.
For horses on pasture, you could get by with little to nothing except for farrier, during those 6/8 months.
Months you will have to feed will cost you at least 150 a month, for hay and grain.
Water and electric will also have to be factor, as if you have to have heater it will cost electricity, and city water will cost too.
You will probably be able to have one horse, on your land for 150 a month, averaging costs out for 12 months. Maybe less, maybe more.
And don't forget that land will have to be kept up, so grass is good.
Horses make me a better person.