I keep 4 horses on my own property and it still isn't cheap. We live in a climate where we have reliable pasture for about 4 months a year. The rest of the year I feed hay and supplement with grain or complete feed. For an average year, this is what it costs to keep 4 horses at home.
Vet costs $300
Extra power and water $500 (for winter)
Fencing maintenance $300
Fly management $300
Insurance $250 - 500
Let's round this to about $6000 per year, so $500 per month for 4 horses I keep at home. That's a minimum of $125 per horse and that's the basic do it yourself, no frills cost for keeping my horses outside, on my own property. I do all my own vaccinations, do not shoe my horses and have not included the additional cost of having acreage property, emergency vet services, trailer costs, riding lessons or riding equipment.
A veterinary emergency can cost thousands of dollars unexpectedly. A simple cut my daughters horse had last summer requiring daily bandaging for 4-6 weeks cost upward of $300 just for bandaging materials. Riding lessons for my daughter are $200 per month if we haul our horse in. To do so requires a trailer and a good truck which both add to the cost. Living on property where we can keep horses and still commute to work adds about a 50% increase to our mortgage if you compare similar homes, one with the acreage and one without. The quad, harrows, mower (or a tractor) and fertilizer/ pasture seed also add to the cost as does a building or shelter for both the horses and hay storage. We are looking into building a basic barn with 2-3 stalls, dirt floor and space for hay storage. Such a building will cost a minimum of $10-15,000 and that is without power and water and doing it ourselves. Even if I spread this over 10 years, a barn adds $1000 or more per year to my total just for the building. I won't even begin to add up what one would spend on riding equipment and tack.
So, to answer the OP's question, YES, owning horses is expensive. It involves much more than a bit of hay feed and / or access to a pasture. Some people can do it for less, but the costs are usually much higher when you really consider everything involved. This is the reason stable board is so expensive and stable owners still hardly make any money. Buying the horse is almost always the cheapest part of horse ownership. The estimate of $1000 per month is not unreasonable for someone who boards a horse indoors.