Welcome to the forum!
I think it's great when adults want to learn to ride, regardless of male or female. I have 2 adult students and I very much welcome them. It's a great change of pace from teaching kids. I especially like that adults ask more questions and the lessons with my adult students tend to have a more free flowing discussion type of feel rather than me talking and me talking some more.
I wouldn't worry at all what others think, it's your dime paying for the lesson. If you are truly concerned, look for a small barn (possibly one with a trainer/instructor that isn't necessarily a boarding barn with a regular crowd) and start with private lessons. Another option if your schedule allows is to go during the day while kids are in school.
As far as what to expect, starting from the beginning you should be shown every detail that goes into it from catching, grooming, leading, safety zones, picking hooves, tacking up, etc. I don't even have students ride the first couple of lessons. It's all about safe handling on the ground, learning to tie a quick release knot, getting a feel for the animal, learning how a horses vision works, etc.
As far as what to wear, can't speak from experience as I'm female but my hubby prefers to ride in boxer briefs.
Life is like a camera. Focus on what's important, Capture the good times, Develop from the negatives and if things don't work out, Take another shot.