There is also the problem of focus. You need to stay focused on what you are asking the horse to do at any given moment and not change for anything, not even the barn burning down. Change your focus and you change your body. Change your body and you change your cue. At a clinic if a person says, "My horse keeps going to the gate." I reply, "What gate?" You need to get so focused you see nothing around you and nothing around you distracts you.
When you give 100% of your focus to the request at hand you will become consistent. When you become consistent you horse will become responsive.
So the horse goes where you don't want, counter move, be the leader by initiating the movement as was stated above, but stay focused. If he goes backward, back him through a shoulder and don't release until he does. He goes left, pick up and take the right shoulder, not the nose to the right. Be specific and stay with it until he does it.
Also, don't go in thinking, 'we'll just walk around,' go in there and do serpentine work, cone patterns, anything that forces you to focus on what you want him to do. As stated above, work him, but work his mind not just his body.