With me, it hasn't been as cut and dry as...do it and get over it! I am anxious every time I ride. I try to hide it well. There is no real reason...just my imagination of what could happen. My two horses babysit me and are awesome. They have never done anything to get me off of them. They put up with me. Any spooking they have done, was never anything I couldn't handle. So what is it?? Who knows, it's just me. I started riding in my late 30's, never as a child. Maybe that is it? I put myself in all types of situations to gain confidence.....trail rides, gymkhanas, trail challenges, riding on the beach, parades......we do it all! But I'm still scared mostly. I always feel better with another person though.
I have a sign on my tack room that states, "Courage is being scared, but saddling up anyway".
Hopefully you find the strength to move on and not be scared. Sometimes, I think it's maybe a part of your personality and maybe something you can't change.