Certainly not an easy situation. You've gotten some good advice already, and I fully understand your predicament.
It is absolutely true that pushing though one's fears is crucial to overcoming them, however as in a psychological "flooding" approach to phobias, this cannot be universally applied.
I'm not a big believer in the opinion that horses are "intelligent" per se. Show me one that does its own taxes and we'll talk. BUT, they are incredibly observant animals that feed off of the atmosphere around them.
1. If you cannot control your reactions and your panic, your horse (any horse) will suffer from this, and itself become nervous and scared. You should not underestimate this mutual emotionality. I've learned things about my own friend's emotional state from watching how a horse reacted to him, despite seeing nothing telling on his face myself. This brings me to my next point...
2. You mentioned that you were an "older" rider. As AbbeyX said, riding should be a pleasure, and fun. Due to the constant rhetoric about confidence and control in magazines and between riders, it is easy to forget THE TRUTH, which is that an animal that weighs 1'200lbs is something that you simply cannot take for granted. I don't mean to worry you, simply to confirm that being nervous around giant animals is totally normal!
That being said, the real issue at hand is balancing between pushing yourself to learn and grow, whilst minimizing risk and the potential for injury, for you and your horse.
The situation at your stable is a mess, let's be honest, but if this isn't changing, then you will have to try and work around it somehow...In the meantime, don't be bullied by your instructor, your peers, or anyone else. Just like our 4-legged partners, we are also animals, and our "gut feelings" and instincts also exist from millenia of evolution, and hence for a reason.
Confidence is a huge part of success, in horseriding, in business, in most things. Your horse will need to feel this from you, and you will need to feel it in yourself to proceed safely. Take your time and be cautious, always remembering to challenge yourself, but AT YOUR OWN PACE!