Take your time. I use self control when I fall of get nervous or scared. Just think of the positives and believe you can do it.
I think it would help if you took short rides on Summer till you gain all of your confidence back.
As for the older lady...some horse people are crazy. As long as your horses are healthy and you love them then I don't think anyone should complain.