It is possible to do it for $300, but you're going to forgo supplements, routine vet care, good tack/tack repair, and even an emergency fund. You're also going to be using a cheap farrier or trimming yourself. If you save a good emergency fund before you buy and are a very involved owner, you may get by, but it would be much better to lease and have the actual owner be responsible for some things. That also frees you to take some lessons, which you'd benefit from, and to take better care of the horse rather than just minimal. And if you get in a bind and don't have the $300/month anymore, the horse can just go home. Once you're riding more, your needs from your horse may change, or your situation could change, so leasing is better in that sense too.
Tug on anything at all and you'll find it connected to everything else in the universe.~John Muir