Anebel is right. If the horse goes when it even feels your leg hanging, you still need to put it on and make him DO something. Eventually he'll understand when leg isn't doing anything, neither should he. When leg is pressed on, he needs to move.
The reason he is so hot off of the leg is likely because you take it away because of his reaction. It's the same concept with desensitizing a horse to a rope. If the rope touches the horse the initial reaction is to move so if you take it away when he moves, he'll keep moving from it. If you stay with him until he shows signs of relaxing, over time the horse learns it's ok to just stand and the rope will go away.
If you don't like what he's doing, redirect him until he gives you what you want. Shutting him down by making him walk will likely frustrate him and exasperate the issue.
"Strength is the ability to use a muscle without tension"