Yep, suspect it may well be just our unconscious bodylanguage, nothing 'fluffy' or mystical to it, but whatever, horses know how we're feeling... often when we're not even conscious of it ourselves! That's why I believe being open & honest with them is the way to go - because if you go there stressed or angry & try to 'leave your troubles at the gate', you might just come across to them as a liar & untrustworthy when you try to 'put on a happy face'.
*Not that that means you should take your anger out on the horse, in case anyone takes it that way!
Horses may just be much more perceptive to subtle body language than us. I remember being "fired up" over an incident right before I got to the barn and found that none of the horses turned out would come near me. My favorite mare went so far a s backing away and nickering at me. Once I cleared my mind and relaxed she came right over and the others stopped staring and went back to grazing.
I have found that "clearing my mind" can be very useful. It has worked for me with the potentially "dangerous" horses. it is not easy in that situation when I also have to be prepared for the worst but not appear nervous or apprehensive. What my conclusion is that by not presenting a defensive or apprehensive attitude, the horse has to rely on what I am actually doing before he decides how he is going to act. it might be different in a situation where I was going to ride a horse that had a problem with being ridden, but most of the time these were horses that I was just taking care of. It's nice when these horses get the attitude of " Oh, he just comes in to pick the stall and give me fresh water, that's no big deal".
I would say that a child, mentally challenged person, or naïve adult who has a liking for horses is presenting themselves to the horse as someone the horse doesn't mind and may actually like having around. I also think that some (certainly not all) horses that know how to be handled and ridden like being in charge at times with certain people. The good school horses and trail hacks know their job better than some of their riders. It may simply be a matter of the rider feeling like a sack of potatoes that is about to fall out of the saddle that makes these horses say "No, we are NOT trotting because it doesn't feel right" that makes them plod along and has nothing to do with concern or looking out for the rider.