Hi, I am new here too and you might want to repost this in the forum on "general/other issues" or something because I bet you will get great advice. Here is what I would do. I would find a horse knowledge person who oculd help me in the beginning to walk me thorugh the process. You might ask at the local feed store if they could recommend someone, or call a local boarding barn and see what trainers they have.
I would be thinking about a) my place, (his new digs). are they "horseproofed"? no sharp things sticking out, no holes to put his feet through.I would walk the fence line to check. I use electric tape (white) because my horses are beavers and will chew wood. I will not use barbed wire. I have had my guys out with no shelter, in stalls with a run,a nd with a loafing shed and I prefer the latter (I think they don't care much about shelter except inthe winter to avoid the wind), but the loafing shed gives me a dry place to feed them,
b) the feed. What does he currently eat? You could ask for advice at the feedstore or call the vet. Lots of debate about it but I give mine daily dewormer (strongid) and sandclear the first seven days of the month but people have lots of opinions here and your vet is the best place to start. He may just get several flakes of grass hay a day and be fine. And then you could call around and see who is selling hay, how much per ton (that's how we do it) can they deliver and stack, etc. Not a big deal at all but when they are living at your house you have to plan it, He may be on straight turn out and not need any hay in the summer.
c) I would get a vet check, not to change yoru mind but just to make sure he doens't have any interesting issues that need addressing. His teeth may need to be floated, etc. Ask your vet for a vaccination schedule, teeth floating schedule etc. ALso ask your vet about a basic "horsey first aid kit" and have her give you the basics of how to use it.
d) "accessories" - which is why a local horse person is really helpful. A water bucket (you can get really fancy and get a stock tank with an electric heater for winter or just a plain ole bucket you wil be chipping ice out of), feedbuckets, halters, leadlines, a hoof pick, curry combs and brushes. you can get these at feed stores, farm and ranch stores, garage sales, etc. I'd get all used tack, and get your horse friend to show you how to put in the bit, etc.
e) get the name of a farrier who will take care of his feet and get you on a schedule.
Horses are SO MUCH FUN to have, you are right they are like kids (somes more complicated!) but totally worth it!
Good luck and have fun!