Personally I take most off except for the brats who know they can run faster and longer then me however there in cheap leather which will snap.
I also had escape houdinis that broke my heart and getting in the gate was an experience ( that was where they were breaking out). We ran electric fencing all round the firled including just in fron of the gate and bolted and chained the gate. However these guys were far from the road.
I would never leave anything on a horse that can be seen from the road as ti makes them easier to rob if they just need to be grabbed.
Ray as regards them getting out keep a bucket of gravel near by and when you shake it sounds like feed they come running.