Some people I know will use a fanny pack to keep essential stuff close and on their person. I have also heard of wrecks when the person needed to come off because the horse was pitching or something and got hurt worse because the belt got hung up on the saddle.
I carry water in a bottle holder than holds a 32 oz Nalgene bottle. Water for the dogs, if necessary in empty 20 oz soda bottles, in my cantle bag. Usually leave the phone in the trailer because most places I ride there is no service. Other stuff, like bandana, Leatherman, flashlight, lighter, snacks, are in my hornbags. Lead rope and raingear in the cantle pack. Pocket knife in my jeans pocket but under my chinks so I can hardly get to it anyway.
Guess I'm not very safe. But in all the riding I do, I've not ever had a problem that happened so fast that I needed any of the above stuff after being dumped by my horse. Actually, can't remember being dumped in the wilderness. Anyway, any situation that has come up, I've been able to get off, get my stuff out of the bags and deal with it.