Once he accepts you holding the whip, the training for you leg begins. You should never have to "kick" the horse the way you say you need to. One small squeeze with your leg should get a reaction to energize. If the horse ignores that, you squeeze harder. If he ignores that, give a light tap...then a little harder tap. Wherever he finally reacts, I give immediate praise to show him what I am looking for. He needs to know that if he ignores you, there will be something else coming.
A dressage whip is an invaluable extension of the leg for a horse who is ignoring you.
Also, if he's finding the impulsion hard, have you tried lunging him with a lunge whip. Be gentle and use your voice, there is very rarely and need to crack a lunge whip. It may help his impulsion and you'llbe able to see if he's working correctly from his hind :) After he's worked well in that, it would be a super time to start desensitising if you decide to use a schooling whip.