I have never blanketed my horses either. However, when the wind chill is high, the temperature is low and the sun is in hiding, the horses go in the shelter. It's mostly the question of wind in the trailer. Maybe I should hook it up and get MDH to drive while I ride in the trailer. Then I would know for sure!
In fact, what I probably will do, is take a blanket with me, JIC she gets sweaty. I can always check her on the way.