Outfit wise? Wear what you want- a t-shirt is perfectly fine for a top (I would get sports ones, I stole my brother's rugby sports shirt to ride in and he never got it back), and for the first few lessons, a good pair of jeans works. For english, I suggest getting breeches eventually. Also, I've heard from multiple sources that it's not a good idea for a man to ride in boxers...
Gear- wise? Generally, I would say that you will need a helmet and a good pair of boots. Some places will provide the helmet, at least for the first few lessons, but eventually you will need to buy your own. You will need riding boots. Some places will let you ride for the first lesson or two in a shoe that has a heel, but I would get your own pair of boots at least- its a safety thing. Try not to hold out when you're buying them- you do not want cheap knock-off boots. "Equestrian looking things" is a style right now, so there are tons and tons of $15 dollar riding outfit knock offs on the market, stuff that looks surprisingly like real riding stuff...
You can order paddock boots (and half chaps, if you're going english) online. I usually use Dover Saddlery - Quality English Horse Tack & Horse Supplies
You could also get tall boots, but I wouldn't do that now. They're expensive and uncomfortable, not to mention hard to fit (at least for me, I have calves like Captain America and legs as short as a munchkin cat... exaggerated but you get the problem.) You only really need tall boots if you're showing in an english discipline. I suggest paddocks. You can find pretty much everything you'll need on Dover, including breeches and helmets. Try to do a little research on each item- look at reviews, etc. If ordering online doesn't work, you can always go old fashioned and find a tack shop in the area where you can buy these things to try on before buying.
If you're going western, paddock boots are okay, but you'll probably want a pair of cowboy boots/ western boots... and for this, I will emphasize: no knock-offs.
There are so many crappy commercial cowboy boots out there that are great for prancing down a side walk in, but if you use them around horses, they're going to fall apart from all the wear and tear and won't protect your toes from any incoming hooves.
I've personally always ridden in Ariats, as they make boots for both english and western and are darn good either way.
Whatever you do, make sure that when you find a trainer, contact them and ask what they want you to show up in for your first lesson and what you'll need to have. Don't be afraid to ask them about anything you're not sure about.