I live in MN, it's cold and windy for a good half of the year, so I've gotten used to dressing for the weather because I would never survive being trapped inside for that long!
-Silk long john top
-silk long johns
-fleece pajama pants
I always have a hat on, it's blaze orange and awesome, even though everyone hates it. Haha!
For boots, I have woman's hunting boots, lightweight and insulated. They keep my feet nice and toasty, however you do need to buy bigger stirrups or your foot will not fit.
Gloves, I layer up again. When I'm riding it's a pair of those thin knit gloves under a pair of those glove/mittens things. You know the ones with the open fingers, but have the mitten part that you can use to warm your hands?