My list of must-haves is as follows...
No Thrush Power
Hooflex Thrush Remedy
Farnam Wonder Dust (THE best for wounds, doesn't promote proud flesh)
Furazone (Don't use under bandages though, just for knicks and scratches)
Vet Wrap and Gauze
2-3 empty spray bottles (you'll always need them for something)
Big rubber bucket for soaking your horse's feet (Rubber is more durable than plastic)
Mild soap for baths and special cleaner for your horse's private bits :p
Detangler for their mane and tail and a good brush that can remove burrs
And last but certainly not least, get some bute from your vet. Bute is like asprin for your horse, it comes in pill form, paste (like wormer), or powder. If your horse ever gets hurt or goes lame, it's nice to have something on hand to make him or her more comfortable. Just make sure your vet explains how to use it properly. You could also ask your vet for a list of things you should keep on hand.
Speaking of your vet... That is another must-have! A solid working relationship with your vet. You don't want to deal with an injured horse and have to go looking for a good vet. Find a good one now, maybe have him come out and do a routine check-up and discuss any worries or questions with him then. That way, if anything ever does happen, he will already have an idea of what he is dealing with.
That's all! Good luck, be safe, and have fun with your new horse!