Wait, I lied, I have more to add.
I lost my confidence two summers ago, when a pony bolted on me and I could not control her. I ran her into a fence and nearly took a spill (I had only fallen two times), and I was scared to death of cantering after that. So, for a year and a half, I did no cantering (unless it was hill-topping). But that is about the time I bought my mare & gelding.. The gelding was too spooky at the time for me to deal with so I rode my mare. She threw me, reared, bolted, bucked.. The whole nine yards.. But I kept getting back on. After I decided she was too much to handle I rode my gelding.. Who is a helluva lot calmer. I just did A LOT of trotting, and nothing else. No jumping, no trail rides, just trotting in a circle.
Well last fall I was taking a lesson on a school pony when the instructor asked me to canter. I was so nervous.. but I did a few strides.. Then each time after I would do more. Then in October (2009) I cantered my pony for the first time. Now I jump, canter, gallop, and everything else on green horses, and my ponies. Do I still fall? Yeah I fell a few weeks ago. Do I still get nervous? Sometimes but I work through the fear.
If you want to continue with riding then just get on a calm horse and work up to the fear. Push that fear to the side and with determination work to riding like a real horse-woman! But you have to want to. If you could care less then screw it, you "know you can't" - so therefore you cannot. You don't have the ability if you don't want to.