Hello! I'm not sure if this is going to help, but I'm going to try. You've got this double whammy situation where you've got this almost (from your perspective) randomly intermittently spooking horse, on the one hand, and then on the other you've got the anxiety inside yourself from having gotten hurt in situations like this - and that anxiety is not easy to turn off. Believe it or not, your brain is
supposed to develop ambulance sirens in response to situations that remind it of whatever caused trauma before, to get you to stay away from such things (like a handsome creamy-coloured 500kg animal that may randomly catapult into you

) - it's a biological survival thing, and it may run counter to what you personally want to do, which is so inconvenient!
So:
You: "I want to work with this horse and get it desensitised and comfortable in a changing environment (etc)."
Your amygdala: "I want this person to stay away from situations which have hurt them before and which I consider threatening to life and limb."
The amygdala does have a point, but you can use your cerebrum to think about making your work with your horse safer so that the amygdala doesn't get another scary experience and say, "Told you so!" and turn up the ambulance siren to even louder next time you work with the horse and get in situations that trigger your amygdala. (And you've been working on that, and you're here to brainstorm things you can improve etc.)
So I see this as TWO separate things that each need addressing:
1) Helping your horse
2) Helping your amygdala
And I've listed them the wrong way around - in terms of order of priority! 😜
A good analogy: If you're in an aircraft and the cabin depressurises, the first oxygen mask you get from overhead should be put on your own face - and then you can get other oxygen masks down to help other people. This maximises your own usefulness in a situation - even though you have to put yourself first (which we've often been conditioned to believe is always selfish, but not so).
Because the problem is, when you're working with a horse, it picks up even the slightest hint of anxiety from you, and insecure horses, or even confident horses unsure about something in particular, will internalise it: "The person with me is nervous, oh no, there must be something dangerous around here!" The ability to rapidly read the moods and anxieties of others, and for the flight response to spread almost instantly through a group, is really important for a herd animal that evolved with predators - it maximised chances of survival.
Basically, you can't work effectively with a horse when you're anxious, or from the point when you become anxious around your horse. Reading anxiety in you (even if nothing happens in a particular lesson) reinforces to the horse that the world is a dangerous place, and it stacks the dice towards another bad experience for both of you. So you need to be ultra aware of when you've got your Confidence Pants on (loved that phrase, made me smile

), and when they're starting to slip down! And be aware that you're trying to desensitise
yourself, not just your horse.
Which I think is the much harder part than desensitising your horse... this is tricky stuff, as
@Knave also related, and it takes patience and awareness and persistence and a whole lot of lateral thinking...working with your own emotions and anxieties is like herding cats, but it can be done.
You've got to become super-confident so you can give your nervous horse confidence. Sounds so easy and logical on paper, but is tricky! So one thing is to forestall and prevent as much as possible situations which become dangerous to you personally when working with your horse, so you don't get re-traumatised yourself and end up with even worse anxiety. This part is probably best as a group brainstorm, and involves lots of things like spending a lot of time with your horse in an area where it is in its comfort zone, and when you build your own confidence back, you can start taking the horse into more challenging scenarios (and I know this is extra difficult because of the unusual spooking patterns of your horse). Also though, consideration of where you are in relation to the horse - there's safer positions and less safe positions when ground handling, for example - and I always lead a nervous horse with me by their shoulder, so I can't be jumped on and should I get knocked sideways hard, at least I will be knocked out of the way of the horse. Ditto - I don't groundwork nervous horses in a halter, they have too much mechanical advantage that way - at the very least I run the lead rope over the front of the halter, like this...
First outing with super-spooky, very hydrophobic horse in 2009. He could turn on a thread and take off like a rocket at the drop of a hat when I first started working with him in unfamiliar environments (and I really had to watch my toes!). Notice that neither my friend nor I ever stand in harm's way when handling our horses in situations like this - we're by their shoulders. No looping the lead rope around hands, use both hands, use gloves if you have to to prevent rope burn, etc. We rode later on in that outing, but walked the horses around on leads first to desensitise them to the new area as a "herd" - it's easier for a person to give a green horse confidence when you're next to them than when you're riding, and usually also easier to stop them getting away from you in a panic. Both my friend and myself specialised since late childhood in re-training "difficult" horses (i.e. reactive, highly intelligent, independent types of animals) for our own use, and ended up with wonderful relationships with our adoptees, who were amazing horses to ride. Sadly both of these horses are deceased now, but I'm just in the early stages of saddle educating a relative of the dark horse above, who also happens to be super reactive and doesn't spook at many things, but spooks instantly at some things, like sudden unexpected noises!
Anyway - I actually had a lot more I wanted to say, like I know that feeling you're describing because I got badly scared working with one particular horse when I was 10 and it was like this mutual reinforcement of incipient disaster - but maybe another time...