Unless you truly startled him - and considering that it was a bath and he was already pushing at you - I think you were too nice. Kicking is A Very Serious Offense (yes, with caps!) to me and "backing up" is not a big enough punishment for something that could have caused you a trip to the hospital or dead.
I think more groundwork is definitely in order and more respect on his part. If you are working with him, his attention should be ON YOU. Where you are, where you're going, what you're doing should be his focus (That's an ideal there, I know no horse is perfect and mine certainly isn't totally there, so don't feel judged.)
I'm glad you're not seriously hurt!