Some we do and some we don't. It isn't out of laziness, it's out of safety. Yes, it is dangerous, but some of them are more difficult to catch and the need to keep one on out weighs the safety issue of keeping them off. I keep them loose enough so that they could (and do) slip them off if caught on something (or they rub). But honestly, you could keep horses in a padded stall and some of them would STILL get injured.
For example, we recently had to put down one of our broodmares (who was halterless) because in a 17 acre pasture she decided to roll and got caught up in a gate. (not the fence) she panicked and broke her neck.
In all that space she had, she decided to roll right next to one of the gates. It was very sad, but what can you do? We saw her only an hour before and she was just standing there grazing.