I took a 10 year break from riding. I stopped when I was 13, and started at 23. When I stopped I was doing 4' jumps, and 3'6 bareback. THe first time I asked for a canter, the transition flung me out of the saddle, threw me so off balance I grabbed the mane and had a death grip with my legs while screaming WHOA and trying REALLY hard to JUST STAY ON.
Within in 4 lessons my flatwork was all good, but when I did my first cross rail my center of balance, and rythm just wasn't there and I ended up clinging to the horse's neck as he carried me to the safety of the fence (what a good little guy)
I'm learning to ride with a "new body", I'm an adult, not a kid, and a lot of muscle memory has to be relearned, and balance regained.
A huge confidence builder for me was doing some simple dressage exercises (flex/counterflex, leg yield, walk/trot/halt transitions, collection, circle work) at a slow pace. I personally found it encouraging, because I felt in control again, and it was opening the lines of communication between me and the horse.
Look at it more like learning to ride again from point zero, and be proud how much faster you're progressing than any other beginner!
Also I found once I have a maneuvre down, it stays with me.
And finally... always end your lesson on a positive note! I know it's hard when you're being hard on yourself about everything. Try just riding bareback for the cool off and just feeling good about being on that horse!