My pony has no problem with that, he's out 24/7, and he has a shelter in the pasture and free access to food, (what do you call it in English? The swedish word is ''lösdrift'') He also has a very long and thick fur that keeps him warm in the winter.
Usually we ride in a paddock (is that the right name?). But for most, we go out for walks in the forest. If you have warm clothes, the cold won't be a problem :) But this week it isn't more then about-1C, so it's quite warm, haha.
Today it was too slippery to ride in the paddock, (ice all over the place), so we clicker-trained/trick-trained instead and then took a walk in the forest! :)