This may sound a bit crazy to people, but I've found that many horses respond off the voice and the emotion more than the actual contact. I've found with students, that to tell them to give the horse a scratch on the neck for praise does several different things that lead to the praise of the horse. The first thing that it does is relax the rider. Whenever you see someone give praise, their whole attitude changes, they are often smiling or laughing, happy, and relaxed. Those are the things that the horse feels and responds to the best. The respond to the tone of your voice and the meaning behind what you are saying more than the word itself.
I don't have any rules as far as praising my own horses. I usually tell students to praise more to relax their own riding and that is the reward to the horse. For any horse that I ride, the reward is in the riding, not in the task at hand. No, I am not that full of myself, but I do work with corrective riding for the horse, where its not so much about teaching them what they can do as far as obedience, but rather how they can feel better doing it. For a horse that is learning the basics, like Keving said, the reward is in the release.