I returned to riding in my late 30's after 10 years off. Being a very confident rider in my younger years, I was very surprised that, after a pretty bad fall, I was unable to get my mojo back. I do think it has a lot to do with understanding the consequences of a serious injury - way more than when I was a kid. What finally helped me was getting a really steady riding partner and by finding 'tasks' to do when I am riding. I discovered that when I went to a fun day to learn about team penning, I was so focused on the cow and forgot to worry about my horse. It really helped me to get my confidence back. I regularly set up obstacles to ride through instead of just hacking around worrying about what might startle my horses. I will even ride barrels at a trot or lope just for fun.
I will say that, for me, although I am not where I was when I was younger, I am enjoying my time in the saddle more than I thought possible after my accident. Good luck to you as you find your way through this.