Welcome to the HorseForum.
Unfortunately I'm going to have to burst your bubble; the initial cost of the horse is really nothing compared to the upkeep.
Hay: Depending on your area, hay can range from $5 to $20 per bale. Buy early and stock up, hay prices are going to skyrocket this winter :) Most horses eat up to 3/4 per bale per day. There are alternatives, such as Alfalfa cubes or supplimenting grain and suppliments for a part of the horse's hay ration. Let's say you buy at $9/bale, which is fairly average, at half a bale per day.. yearly total: $1642.50
Grain: I don't like to grain without a reason, but depends on the type and frequency. I'll assume you might grain once in a while, say one bag every 2 months = $180
Treats: $20/bag every other month = $120
Farrier: since it is only for one or two horses at a location, you're probably looking at a call-out fee (paying for mileage) on top of the farrier's regular fee. Looking at around $30-$100 per trim, every other month, we'll say $50/trim = $300 (PS. this is for a barefoot trim, if you're looking at shoes, at least double this amount.)
Deworming: if you do it yourself instead of getting a vet to do it, it will save you money. Dewormer costs around $20/tube, and needs to be done about 4 times per year, but that is something to talk to your vet about. $80.
Vaccinations: You need a 4- or 5-way done annually, at least. The West Nile and Strangles vaccinations also come very highly reccomended. Each vaccination comes with a flat fee, as well as an administration fee. Looking at my bill, each vaccine cost around $45 on average, plus the vet call-out fee. $135 plus call out fee.
Floating: Needs to be done annually. My horse's last floating cost $160 plus vet call-out fee.
Other: I always strongly recommend keeping a certain set amount set aside for any emergencies that may arise. It's up to you as to how much of course, but I strongly recommend at least $500 set aside... that can take care of a down payment on an expensive surgery (that you can make payments on) or can take care of euthanization and removal of the body if such an event should arise.
Total: (not including emergency funds) $675
Fencing: search around for fencing costs (NOT BARBED WIRE OR SMOOTH WIRE), but here is a good company: http://www.systemfence.com/index.html
You can send off emails to each company that you find and see how much they'd charge to fence your land.
Shelter: This is not an option, horses need shelter. You can build the shelter yourself or buy one pre-made. Again, just google Horse Shelter, and fire emails off!!
TACK and TOOLS (I am assuming you're riding English, let me know if you are doing western instead)
Saddle: A good used saddle is anywhere from $500 upwards. A good new saddle is at least $800. Each can range up to $5000 or higher. $800 is a good number, so I'll use that.
Saddle pad: Again, ranges, but usually $25 for a nice cotton pad.
Bridle: Huge range, but usually $125 for a fairly nice model.
Boots: Protective boots are a must if you're doing anything high-impact. $40 for fronts only, or around $80 for front and back.
Halter and lead: $30
Grooming tools: $50 for everything.
Alrightey, this is a very basic and crude list, but my list is a
GRAND TOTAL of $3852.50*
*without fencing, shelter, or emergency vet costs. Oh, and this is without buying the horse.
Most people easily spend over $5000 per year on their horse.
Again, this is a very crude list, I'm positive I'm missing some costs... you have to factor in the little stuff too, like horse blankets (if you blanket), fly spray, water buckets, grain buckets, salt licks, bits, trailering, etc etc.
Best of luck.