You said that both times you fell, you tried to hold onto your horse's neck. To hold onto my horse's neck, I have to lean forward. If you're in a position where your horse is spooking or getting too strong, forward is the last place you want to be. You want to be sitting tall, deep, square with your heels sinking into the earth so heavily they'd leave a dent. I would suggest reading the two Sally Swift centered riding books. There is a lot of good insight there about creating a solid base of security in the saddle. If you go to any horse show or training event where there are a lot trainers around and a horse gets out of control, you'll hear every trainer simultaneously shout "sit back". Fetal position will get you off every time.
You just have to see your distance...you don't have to like it.