The horse is anticipating. Pulling him into the ground will not change that. All you will end up with is a horse who will start scotching and not wanting to run.
I have a horse who does not so much anticipate but she knows her job and loves it so the min. You put your hat on she knows it is show time and she runs the pattern like she is going for a 74. Which is great for my trainer not so much for me.
Anyway. When you have a horse like this or anticipating really the same problem at the end of the day is keep changing it up. When they speed up change direction. Do a lot of standing and boxing of the horse. If the horse learns that the hat means shows or running the long side of the arena means fencing or stopping then you MUST change things up.