I would go volunteer at a local stable. You'll learn TONS of stuff in a short amount of time (I don't know how much you already know). That way you'll atleast have basic knowledge in pretty much every area.
And actually being around the horses and applying what you're learning is A LOT more helpful than just reading it in a book or on the internet :)
HINT: The price of up-keep will be more than what you pay for him!
I'm not sure how experienced with horses you are, so all this information is based on thinking you are a beginner to horses.
Find a reputable horse farm that you can take lessons at with your horse. That will take care of your riding.
You also need to have a veterinarian that can come out to your place or you take your horse there. He/She will help you to take care of your horse's health.
Make sure that you have a place that you can go get hay for your horse. Good quality hay is essential to your horse's health. Talking to your veterinarian will also help you decide if your horse needs grain.
A farrier will also be needed. This person will trim/shoe your horse.
Try to set aside at least $100 every month for use in an emergency. This emergency will likely be health related.
Volunteering at a stable is your best bet of educating yourself and getting hand-on experiences (the best kind). It might also be a wise idea to keep a binder or something dedicated completely to your horse, organised with phone numbers, directions if need be, what you are feeding him/amounts fed, perhaps a medical history, vet and farrier appointments, etc.. This can be handy.
I never thought of volunteering at a stable when I got my horse...what a neat idea =)
But, as others have said...I have no idea how much you already know, so I will begin with this:
Every one sais horses are a lot of work and you, (at least I did) reply "Oh, I know"......no you don't lol. They are alot of work and it takes alot of time to get accustumed to your horse, and you to them before you can TRULY understand the horses' needs despite what information you get from previous owners. Your horse will probably not truly trust you right away, so alot of ground work such as GROOMING and taking the horse for a walk around the pasture every day, are two GREAT ways to get to know eachother and create a bond that will be needed for the future. I would do this before you ride for the simple fact that riding takes a bond between horse and rider: and though you can "ride" any horse, it takes a true bond between the two to truly make magic happen, even if it's just on a trail. Lessons, as previously suggested is another great step. I say ALL of this by experience. A first horse is so exciting- you want to do everything at once! I'm so happy for you =)