I started at 50. Since I was stupid enough to buy a horse and THEN start learning (bad idea), I was able to ride 4-5 times/week. A couple of things I found:
1 - 40 years of jogging left me with very tight hips. Tight hips make it very hard to relax into the saddle. It keeps your thighs squeezed together and shoves you out of the saddle. After 4 years...I still have problems. Rode yesterday, and had cramps in my hips near the end of the ride.
2 - Heels down. Often preached, but I honestly think older riders deserve some slack. When people told me to put my heels down, I tired to SHOVE them down, creating tension thru my entire leg and making my riding much worse. If you have to choose, and a lot of older beginners do, a relaxed leg is far better for good riding than heels down.
However, it will get better with time. This is one area I've made progress:
I'll also note that it is far easier to get your heels down if your feet are slightly forward of your hip. I shoot for the back of my heel aligned with my belt buckle.
3 - I was also told toes front. Right. Like that will happen. After 4 years of riding, I'm happy if my toes are only 45 deg out. Trying to get my toes further forward again results in tension in my leg.
My favorite book on riding is "http://www.amazon.com/Commonsense-Horsemanship-Vladimir-S-Littauer/dp/0668057912/ref=sr_1_1?s=books&ie=UTF8&qid=1352064478&sr=1-1&keywords=commonsense+horsemanship+littauer". He argues the key is to learn to feel your horse's movement and balance, and to move in harmony with your horse's balance. In my experience, too much concern about my 'position' was harmful to learning to feel and move with my horse.
I had a back injury (thank you, Mia, for the bolt that led to it!) that is only now, 4 years later, starting to work out of my lower back. I didn't realize at the time how much that limited my progress. I didn't start cantering for 3 years, but then, I didn't take many lessons either. And cantering with a very stiff, rigid lower back wasn't going to work anyways, as I look back on it.
In my limited experience, most younger people really get a kick out of an older guy learning to ride. I've met with lots of help, and have yet to have anyone try to discourage me. My youngest daughter and I ride together regularly:
Yes, that is Mia...our current goal is for Mia to become a good trail horse. We recently started on solo rides in the desert. Riding is a lifetime exploration. I enjoyed watching a video on campdrafting, and in particular the competitor who was 80! Paradise Lagoons Campdraft - YouTube
Also - I think western riding is more forgiving for an older beginner. I've become a *******ized rider. I ride an Aussie-style saddle with a forward seat, except with longer leg than normal and use western reins. The western lessons I took for about 4-5 months were very good. The instructor was very big on teaching how the rider's position affects the horse's balance and willingness, and that is very important no matter what your riding style.
Good luck! In 5 years, you'll be as old as I was at the start...
Oh - holding breath. When I'm nervous riding Mia in the desert, I sing. She doesn't mind, and I have a hard time being tense while singing "Red River Valley". Or "Four Legged Friend":