I've had this happen twice. One was a respect issue and the other was a trust issue. The first one was a filly I got who had a 'tude. I was later told her mom also had a similar 'tude and that is where she got it. My husband still has a scar from her. However, since I was the one who worked her, round pen her, etc - she learned to respect me. However, this respect did not automatically transfer to other people and thus each person who worked with her had to earn her respect by themselves in order to be able to get her to work with them properly.
The second one is one of my current horses, Willie. He came to me as a unhandled stallion with serious trust issues. It has taken a long time to earn his trust and I am finally gaining it. However, he doesn't trust others. He does seem open to other women over other men - especially leery of big men, no no one can just walk out and catch him like I am finally able too. However, the more my husband does with him (feed, pet him after I caught him, etc) he also seems a bit less jumpy around him. I believe with time and just introducing him to others the issue will eventually go away.
In both cases - having other people work with the horses is a big help in getting over the one-personness issue.