I think it depends on the horse, but it clearly is possible. My gelding is very passionate about collaborating with me, but he turns into a very different - an extremely lazy one - horse, if any other rider mounts him, even if a very skilled and experienced one. He's just one of those who tend to bond with one person only and won't open up for others unless he trusts them to an extreme level. Which he doesn't.
I have come a long way, to surrender my shadow to the shadow of my horse.