It wouldn't completely shock me if the horses were fiddling around with the latches and such, but it definitely does sound fishy, especially the bit about the sapling being run over.
I would at least install a padlock on the tack/feed room door if possible. Locking the entire barn or the stalls solid could be a hazard if there were ever a crisis and you needed to evacuate the barn quickly, but at least locking the tack area would ensure that your gear and equipment aren't damaged or stolen, and that the horses can't get into anything dangerous if they do get loose. Also, if someone manages to get through that, you know it wasn't a horse, the wind, or a cat.
I like to pop a spring-loaded clip or carabiner through my stall latches -- those are quick to remove with one hand in an emergency, and I have yet to have horsey lips figure them out.
Definitely put up some cameras -- game and trail cameras are getting more affordable by the week, and run on batteries with removable memory cards. Most are motion-sensitive for still shots, although some of the more expensive ones will take video as well. You may be able to find better pricing, but there's a pretty broad selection in the link below. If you have any outdoorsy friends/neighbors who are into trail cams and game watching, you may be able to borrow from them for a few weeks and at least find out if you have a team of horsey Houdinis or a two-legged vandalism problem. Trail Cameras - Game Cameras - Deer Cameras : Cabela's
I'd hang one at the entrance to your barn to capture anyone coming or going, and one facing the latches that keep getting undone to capture how it's happening. Make sure that they're positioned in such a way that, if it is vandals, finding them and getting them down to dispose of the evidence makes enough noise and trouble to attract attention.
There are also wireless driveway alarms -- position a motion sensor by the barn door, and keep the speaker in the house to let you know if someone/thing is coming or going (I don't think you said if there's a residence on barn property or not... won't work over an extreme distance). The main problem with those IME is false alarms... I can't count how many times I've had barn cats and raccoons trip ours... :P
Good luck! Keeping my fingers crossed that it's only some very clever animals at work!