I can totally relate to this. I had a few years of a break without riding steadily, and the first time I got back in the saddle I took a fall...at a /walk/ >_<. It shattered my confidence in myself. Haha I had lost ALL my balance, and didn't expect to fall. But luckily I have a friend that stables with me, and I had the right horse to reteach me everything. I owe her most of the credit. If not for her calm, laid back and easy personality, I think I'd still be scared and unsure.
A couple of months later, I'm almost back to my full skill and am SO happy with the progress. Keep up the work. The outcome is so rewarding.