It's hard to forget a fall! My rule is: Never let fear make you afraid to live! We don't have a lot of time, and I don't want to have any regrets!
I never want to say "If only....". Don't sell yourself short - you have had some unfortunate experiences, but from here on it doesn't have to be that way! You could start with ground work - that helps you gain respect from your horse - I like Clinton Anderson, but other trainers use the same principles. I still work my horses from the ground - it's just plain helpful.
The harder you work at it, the more you will achieve and it will also boost your confidence! Maybe working with a trainer would help you rebuild your confidence, too - working in an arena and having someone there with you can help you to focus on improving your skills - they may be rusty, but you do have skills. You certainly have the desire - start building from there! You watch your friend ride your horse - THAT COULD BE YOU!
Start as slowly as you want - baby steps are fine! You want to ride your horse, and you will - think positive thoughts - you can't do anything about the past - put the negative thoughts to rest! Time has passed and it sounds like you trust your horse - that's a great beginning! A friend of mine used to go to McDonald's before we'd ride -- this is the TRUTH -- she would get a hamburger - a carton of milk and she'd take 1/2 Benedryl tablet BEFORE every ride to sooth her nerves!!!! She had a wonderful horse, but she would get herself so worked up and she was afraid her horse would pick up on it! That did help her -- today she is McDonald's/Benedryl free and she enjoys riding again! Might work for you!? You are almost there.....best of luck and COWGIRL UP!!!!!!!