If anyone rode my mare without my permission, baby Jesus coming down from heaven could not save them from my wrath!! No way, no how, would that be acceptable. Nobody touches my horse without my consent.
I would move barns a.s.a.p. There is no excuse in the world they could possibly make that would be acceptable.