No, I understand where you are coming from. I just have a friend who adopted a horse out. The horse moved states and she was like that's fine. Well she ended up in the area for a show and was told about this starving horse. Starving horse lo and behold was the one she adopted out. She had a contract and did manage to get the horse back. I think unannounced visits are very close to trespassing. Than again, I have heard about folks who adopt out horses and end up being told the other owner sent the horse to slaughter. Most of them have a contract because they feel it is necessary.
You may not like their contract, which is fine and sometimes I think they go overboard. At the end of the day, they are putting it all on the line. You know going in what brand of crazy you are dealing with and you have the ability to say you don't agree with it.