Yeah, horses are definitely not cheap.
If you can keep your horse at home, you'll still need to buy feed each month, which depends heavily on how much grazing the horse gets, hay prices in your area, if the horse is an easy or hard keeper, etc.
Farrier is anywhere from $40-120 (as Analisa mentioned) every 6-8 weeks. And this is something you cannot cheap out on.
Minimal vet care is going to include yearly vaccinations, dental float, etc. Odds are you're going to need additional vet care, especially as a new horse owner, as you'll freak out about every little thing (I just got my first horse 3 months ago, and even though I did a full lease before and dealt with some minor injuries, I've still asked my vet to look at my horse like 3 or 4 times since I got him! However, she does board at my barn and her office is less than a mile away so it costs almost nothing to have her out...)
There will be A LOT of one time expenses for tack and equipment. You need to make sure the saddle fits well; you can save yourself a lot of hassle by just getting a saddle fitter out for your horse. The cost is generally small compared to what you'll end up spending on the saddle, and well worth it. You'll also need a bit & bridle. You may need to try out multiple bits before getting one that works for the horse.
And there are endless small supplies that you need and which sometimes unexpectedly need to be replaced. Brushes, buckets, shampoo, first aid supplies, thrush treatment, saddle pads, fly spray, blankets, hoof pick, gloves, halters, leads, lunge lines, surcingles...
And then if you decide you want to show?
Horse ownership is definitely not something to be entered into lightly! I'm lucky my DH is still supportive of me after he sees all the stuff I keep buying