My rule of thumb is, if the temperature is 20+ celcius then it's fine to bathe.
If it's more like 15-20 celcius then I use a cooler to help them dry faster.
If it's more like 10-15 celcius and not sunny I probably wouldn't bathe, maybe a quick rinse on the saddle area after a heavy ride, and use a cooler.
If it's 10 celcius and below I wouldn't.
I find, also, to help a horse dry faster, to lunge them after you bathe/rinse. Basically if you're not comfortable naked and wet outside your horse probably isn't either!!