Using chainlinked fence or cattle fencing (kind of like chainlink but without the metal sticking up over the bar) would work for a goat. I just wouldn't put anything they can climb on right next to the fence because chances are they'll climb it and hop over. Goats are pretty cool though because they'll eat weeds that are around the house. They are garbage guts. I'm the same size as you and I fit very well on my 13.3 POA. So I think 13.3 to 14.1 would be a really good range for you. Maybe a little bit of an older horse (10 years would be good) that is broke for trail riding and will ride out alone.
If you don't go with a goat, a mini would probably be better then another pony due to the size of the area they'd be in.
Where do you plan on storing hay and grain? Tack?