I've only ever touched him with the whip once, last summer,
I'm not trying to offend you, but you said you only "touched" him once last summer. That sounds like you either just use the whip to threaten him or you tell me what you use it for?
Loosie is right. I do not like whips. I do not think they are necessary 99% of the time. I see probably 20 people misuse them and get the wrong response (like the anger you are seeing) for every person that gets the correct response and gets respect. If a person is not prepared and able to use a whip correctly or is going to quit when they get the wrong response, they are better off not using one at all. JMHO
If you are going to use a whip around a horse, you must first get the horse to understand that the whip is not going to hurt him and that he MUST move from it WHEN ASKED (not before). So, the first thing I do with a horse that does not respond correctly to a whip is teach it to not fear the whip.
I will hit the ground repeatedly and hard all around the horse. I will keep it up until the horse accepts this. If it is really stupid about whips, I will use 'approach and retreat' until it is cool with the whip. Then, go to hitting the ground. Then, I will rub the horse all over with the whip. Next, I teach it to move when I tap it with the whip and smooch. One session usually does it with a short refresher the next couple of days.
I teach a horse to move from a whip like I do about every other lesson I teach a horse. It is really as simple as 1, 2, 3, 4.
1) I make sure the horse is ready and able to do what I am going to ask.
2) I make sure I ask in a simple, plain and concise way so the horse has no doubt what I am asking it to do.
3) I will ask twice, being more firm the second time.
4) If the horse does not do it, (or make a decent effort) I make him wish he had listened the first time I asked.
When you train a horse effectively, you teach it very early that it needs to listen and it needs to respond correctly. The sooner you establish this relationship, the less they argue and 'push back'. They know nothing you ask is unreasonable and they know compliance is not optional. That is why we never have a problem with horses loading into any trailer, accepting about anything we want to do and they try really hard to do what we want. They just never get the chance to argue with us.
Oddly enough, they are a lot happier, too, when they know exactly where they stand and that they need to do anything and everything we ask. I cannot even remember that last time a horse tried to bite, or threatened to kick or just stood there when I asked it to move. They love structure and 'sameness' and want to know who is in charge. They really prefer it when we are in charge. They crave a strong leader and not an equal.