I agree for the most part with Spastic. However, what you need in your first horse will also depend on what type of riding you intend to do. Riding lessons are practically a necessity if you plan to ride English. If you plan to ride in a western saddle, lessons are a good idea, but not an absolute necessity - unless you plan to show or compete in something.
I've never had a lesson - not a formal one, anyway. But - I'd love to have lessons now. Just can't find anyone who has a suitable horse and saddle for me. (I'm super plus size!)
Your first horse should be older, calm and well trained in at least the basics, with no bad habits or vices. Around here, you can find horses like for $250 - $1,000 (or more, if you have lots of $$ - I don't), depending on what their breed (and sadly, color) is. A nice, grade chestnut gelding sold up the road from me for $250. I'f I'd known he was for sale, I'd have snatched him in a heartbeat for my grandkids!
Plain Old Dee, horses Dancer and Rain I believe in dragons, unicorns, good men and other mythical creatures!