Hey! You seem really dedicated, which will only bring you good things in life. I'll give another Australian POV, because the costs and responsibilities involved in owning a horse are really VERY different between here and the US.
I was like you, but younger. I begged and begged and begged my parents for a horse - My dad would always say 'she's never getting a horse' - Now he drives me and my horses all over NSW every weekend :]
Costs will vary greatly depending on what horse you buy and how you intend on keeping it. I'll stick with the cheapest options for now, if you want info on the more expensive options, just let me know!
Okay - So first you have to find the horse! I concur with another poster - Start buying horsedeals! Look through every month - I circle all the horses I like with highlighter. Maybe pick some out you like that are in your budget and show some horsey people - They will be able to tell you why or why not that horse would suit you. The more you know about the market and buying and selling before you do it, the better! Don't stick to one breed or colour - Have a look under Allrounders and Pony Club - There are always good first horses in those sections. For a first horse that is safe and fun you would be looking at anywhere from $2,000 up to $8,000 - More if you want one who is proven at a specific discipline.
Now - Where to keep it? Depending on where you live, there will be different options. I live in canberra, and we have government owned agistment paddocks all throughout the city - this is where the majority of people keep their horses. You get a paddock for your horse, shared with a lot of other horses, a yard complex for grooming/washing/saddling, hospital yards, and depending on where you are maybe an arena and some jumps. For this, you pay about $20pw. You are responsible for your horses care - There is a whole paddock worming schedule and paddock roation schedule.
There are also private properties who offer agistment - Often you can choose between a private paddock, shared paddock, self-care stable, and full-board stable. The cheapest is again a shared paddock which is entirely self-care - they run at from anywhere from $18pw up to $40/$50pw, depending on facilities. Next up is a private paddock - you don't share with anyone. Then a stable - Normally a stable and a paddock, but still self-care, and then the rolls royce of agistment, full board. Full board is when your horse is taken care of by the agistment staff - Fed, rugged, brought in at night and put out in the morning. I don't know prices on those options as I have never used them :]
My horses are paddock kept where my dad works - We get it for free but two friends have their horses there and pay $22pw.
Now, feed and care... The ongoing costs :]
These are very dependant on the type of horse you buy. To keep things cheap, I would recommend NOT buying a Thoroughbred - the majority of them are notoriously high maintenance. You would be better off with a pony or a tough breed such as an Australian Stock Horse, Arab, etc.
I have two arabs and an ASH. They are very low maintenance.
Feed - Most of the year, I don't feed. I supply the horses with an equine mineral block to supplement anything that may be missing from the pasture - But apart from that they get all they need from the paddock. Of course this is dependant on having a well managed place - Paddocks rotated so that they don't get depleted, weed spraying, etc. When I do feed, mostly in winter, I don't feed much. Hay will normally be nough for most horses, even if they are doing a decent amount of work. You can feed meadow or lucerne hay - Lucerne hay is higher in energy and protein. I try to feed meadow when possible - It runs for about $18-$23 a bale. In winter we give one or two biscuits a day, per horse, only if they start to drop weight. When I am competeing, I give a feed of Speedi-Beet and a little Copra - One bag of each lasts me months on end.
So really, if you buy the right horse, feeding isn't a massive expense. A mineral block and hay is all the majority of easy-keeping horses need.
Care - Feet. Another advantage of buying an easy-care horse is not having to shoe! All my horses are barefoot. They get trimmed every 6-8 weeks, at $30 per horse.
Worming we do when we think of it, but most agistment places worm every few months. Worming paste is about $10-$25 a tube.
I get teeth done every two years or so unless there are problems. Our dentist is about $60 per horse.
We don't get shots - Our choice. So I don't know how much shots would cost. We have never had strangles and we only give a tetanus shot if the horse injures itself.
Vet bills - Now, it is so important to find yourself a good vet. My vet is acountry lady who works out of her garage - She is a great vet and charges extremely reasonable prices. She is actually in it for the animals, unlike most commercial vets. My horse recently had crystals form in his urethra - 3 farm visits, numerous shots, and then he went and stayed at her house for three night - $250. If I had of gone to one of the commercial vets around here, it would have been well into the thousands. It is all about who you get.