Changing pastures won't help YOUR relationships with him. Unless you'll live in same pasture and behave as alpha mare.
Just be consistent with him in what you do. If you don't have much experience dealing with such horse (and no offense, but it sounds like it) get a help - trainer, who will point out what's done incorrectly and how to position yourself. It's really hard to tell or advice anything when there is no info provided.