it definately depends on where you are and what you want the horse to do. i am in pennsylvania and i do hunters. i am looking for my first horse right now and my budget is about a $5,000 purchase price and thats very low. i need something safe, young, and something that looks nice to show in the hunter ring. my budget should technically be at least twice what it is now - but im a college kid trying to do this on my own so thats not likely to happen right now haha. this will be able to happen though because i can also get something kind of green, as long as its safe.
in my area, if i was looking for like a dead safe trail horse, i could find something descent for like $1,000. if i wanted something for me but just not be as fancy, i could get like a $3,000 horse or so. it really just depends on what you want the horse to do.