I will Never leave a halter on in the field or in a stall. I have personally known too many people who did and their horses got hung and died or they caught a back foot in the halter and injured themselves.
There are so many reasons why not to. First horses are accident prone idiots and they will find a way to get hung up in them, even if they are breakways. Second the halters WILL mark their faces and leave rubbed patches, not nice.
I had a poor 25 year old mule brought to me one time and the idiot people had the halter on too tight. It caused a sore which then got infected and then infested with fly/gnat larva! When I got her, her neck was swollen and I started treating immediately with everything known to man, but she was too far gone and we lost her. THE WORST REASON why you should ALWAYS take the halter off.
Off. My horse is great about being caught most of the time, so I see no reason to leave his halter on. If you do leave the halter on make sure it is a leather or breakaway halter so if it gets caught on something it will snap before your horse can hurt himself.