What is happening is that your horse gets "stuck" in reverse, and doesn't know how to come out of it in a calm, confident manner. In some horses, when too much pressure is applied beyond their ability they can get stuck, and the only way out of that place is to explode.
What I would do is before you ride the next time, just leave your horse in the halter. Put him on the rail of your arena, and back him one step, and just see what he does. A few things might happen.
If he continues to back up on his own, quietly walk along with him. Do not correct him by asking him forwards. Just hang out with him until he realizes you are not asking him to back up. Do not have any pressure on the lead rope, make no corrections if possible. Wait it out until he stops backing up on his own. Once he does, pat on his face and then ask him forwards by walking backwards yourself and drawing him to you. Stop, pat him, then ask for him to back up again. Repeat this until he realizes you are only asking one step from him. No matter how long it takes, do not correct him from backing up "too much". You will be able to work up to one step back, one step forward, one step back, one step forward.
You need to teach him how to come out of backwards. Doing it on the ground is safer for you, and more clear to your horse because you arent dealing with habits and patterns that you have already created.
If your horse backs up and springs forward like its his only job in the world, with no relating issues, then add a little life when you are asking for him to back up. Once you have raised his energy level, stop immediately and allow for himself to come back down and get with you. If he does not, and instead backs up too much, repeat as above. Adding life and energy will train him that when he gets to that state of mind, where this is more pressure, that he can calm down and quietly remove himself from being afraid.
Take your time. Be patient. Give him a chance to work it out.
In riding, a horse's energy is like a river- guided by the banks but not stopped by them.