My mom is very much the same as you are. She rode when she was younger and was pretty fearless. When I started taking lessons, she did too and was jumping 2' courses while I was still learning to canter. (she started her lessons long after I did). We bought our first horse, a 16.3hh appendix who was a gentle giant and she rode him in lessons all the time. I was scared of him cause he was huge and I had a few bad falls off of another bigger horse that was too advanced for me. We were out riding him one day in the indoor when someone walked past the door outside and spooked him while my mom was dilly-dallying around, caught her off guard and she fell off onto the wall and hurt her back. Since then, she's been very nervous about riding. She would ride an older quieter western mare from time to time, then get on Toby and just walk with maybe a little trot. She never really got over that fear.
I have my own anxiety too, usually going into the show ring. At home im pretty ok, I am scared to death when my BF is driving his mare...not sure why, it just freaks me out that something will go terribly wrong. Theres all sorts of little things that make me nervous, stupid things too that I shouldnt be nervous about. Then once I do it, I just laugh like 'really? I was nervous about that?'
I think we all have a little anxiety about something or other when it comes to horses, and that's part of why we stick with it. Cause if we can get thru it, its like an achievment of sorts.