Interesting topic. Horses do have wonderful memories, however, like gaited mention, they don't have memories like us. They are extremely forgiving, but not in a way that we would like to think. Horses are very specific in their thinking, they all tell their story in their own way, and its how we deal with it that makes it better. Its important to remember that the horses first priority is safety. Its not a solid stall and a good fence that makes a horse feel safe, but instead the comfort that the horse finds in its herd members and specifically the herd leader. If the horse has had a past in which people have always been the predator and never the herd mate, then that has to be the first step to overcome. Very often, I see people who can make that horse lose the fear of people, but in the same way that I don't fear a kitten. I'm not afraid of it, but I'm definitely not going to let it be my leader. To be able to transition between the invitation into the herd and then placement in the herd with people is where many people may miss a step. Its not about sympathy, but about offering something solid. Its not sheltering, but teaching how to accept and enjoy the world. The mental health of a horse is something often ignored by imposing our idea of safety on them.
Horses are also specific with their memories. One horse that I had come in for training came in with a laundry list of mental issues, not entirely strong physically, and his owners couldn't figure out why he kept throwing them. He had connected the owners nervous energy around her children, and he became nervous around children. He had been mistreated in the past, and one of the first things to teach him was how to handle corrections without overreacting. I never sheltered him, nor did I baby him, but I did lead him. That was what he needed.
On another note, I do think that we tend to overlook some of the reasons behind the defensive behaviors. A horses behavioral issues won't only be seen as what we see as mistreatment through a heavy hand. I have seen people think that defending their personal space as abusive, but not seeing their horses insecurity, a poor fitting saddle, or a poor riding style as being harmful to the horse. A physically healthy horse is much easier to acclimate to change than a physically damaged horse. I think something sometimes looked at as "poor conformation" is often overlooked as poor posture that is affecting the way that the horse is going to react to stressful situations due to the strain on the fight or flight reflex. Many horses I work with are a combination of fixing physically at the same time as mentally. Another thing that we take forgranted is that just because the vet says that something is healed, learning how to use it again is a whole 'nother story. Horses are masters at disguising any kind of deviant movement. Unequal distribution of weight between the legs can cause many musculoskeletal issues that many people can't even see, they only see the behavioral problems associated with the horses defensive wall going up as its body is compromised.
So, to answer the question, yes, I pay attention to everything that my horses are showing, as well as every other horse that I work with. Its not about avoiding the memory of pain, but instead recreating it, and figuring out just what kind of pain it was, be it emotional or physical. Is it something they are hiding because they still see me as the preadator? I consider it my job to learn as much about their body as I can so that I can recognize what lies between the lines. Often times, only working on the mind can cause the horse to harm itself more as it only learns to escape any kind of pressure applied by the leader. When a horse comes to my barn, they are not my pet, they are not my friend, they are not some poor soul that I need to take care of, they are my family. They will need to rise up to a certain level of expectation, but I have no problem showing them how to get there, regardless of their past.