You are probably right. You think with more time in the saddle and more time with our specific horses it will help?
If you're most comfortable in an arena, I'd ride there and get as much time on the horse as I possibly could. On days when I couldn't ride, I'd spend as much time as possible grooming and just "being" with him. If you're more comfortable out on trail, same advice. Start in your comfort zone.
At 40, you now know what a bad fall could do and understand the consequences to your abilities to parent, cook, clean house, earn a living. As a child, all you thought about was riding. It's a whole 'nuther mind set.
I have a friend, about your age, and she bought a green 2 year old with an awesome temper and mind and put him in full time training with the same guy I use to break my youngsters. She's had him 4 years now and still cannot relax outside of the arena and she's not all that confident in the arena. She knows I love to trail ride and asked me to pony her a few times, so that if the horse decided to get silly, someone else would already have hands on. The horse never even thought of putting a foot wrong, but SHE was the spooky one. If she'd have been a horse she'd have had that white eyed, flared nostril, snorting, jigging thing going on the whole 2 hr ride and would have gotten home completely lathered. Her horse, however, was utterly bored with it all. We did it 4 times and came to the conclusion that some folks just don't need to leave their security area. She now rides in the arena, 4-5 days a week and is making steady improvement in her skill set but not so much in her confidence. I totally will not pressure her to come out on trail with me again but would take her if she asked me to. Give it time and repetition and "sweaty blankets" for both of you and see if things improve in 6 months. I think you'll find you've gotten a lot of your natural confidence back. My friend was never a confident rider, not even as a child, and had 20+ years off when she didn't ride at all. It doesn't sound like you have that kind of lack of confidence, just more of a 40 y.o. consciousness of what could happen. Just convince yourself that it most likely won't.