That sounds like a perfect solution!
As far as companion options go, I have a pair of goats that are my girl's buddies. They all get along fantastically.
I was really lucky with mine that I was able to get an older doe (9 years old) and a young wether as a pair (I really would have preferred 2 older goats but this works out just fine). The doe keeps the wether in line and since she knows "the rules" (no escaping, being polite, the whole thing) she also makes sure he follows "the rules". If I had gotten 2 young goats...I can see myself hating goats.
Anyway, they're SUPER cheap to keep. They don't need anything really beyond what I'm already getting for my horse (already had a 2 stall barn so they "own" one stall and my horse has the other), I do feed them COB which I would never feed to my horse but one bag lasts them about 3 months...etc. I think in the nearly 4 months I've owned them, I've maybe spent $150 on the goats, including initial fencing. Hay prices would probably increase that $$ if I were to be really nitpicky but it's would be maybe $3 for the pair/week. I don't think they would even come close to finishing off a single 60lb bale of grass hay in a week (with grazing, of course).
And these guys are large goats!
They do need regular hoof trimming like any hooved critter but it's something that's pretty easy to learn to do yourself and it's a VERY easy job.
Also, mine both have horns and they really don't cause issues. Of course, I've been careful from the beginning to not handle their horns too much and to chastise them if I do end up being touched by horn, but it really has not been an issue. I guess it would possibly be different if the horse in question was very physical with the goats but my girl will nip them if they get too frisky with her (she's definitely the boss! Haha) so they pretty much give her her space.