When my horses behave, I praise them and sometimes, if we were working, give them a moment of rest. But really, it depends on the horse. My arabian, Envi, is a total sucker for scratches. So for a reward, he gets his cheek scratched or his neck or ears. My other horse, Gerronimo, is more of a hugging type of guy. Yes, I do reward him with hugs, and yes, he likes it. He's very sensitive, lol. But every horse has a 'reward'- some like pats and scratches, some are happy with a soft word, some are happy with not working for a minute- my sister's horse needs boundaries set quite clearly, so no hugging or scratching for him. I'm not big on treats as rewards, but I do use them occasionally, like with my sister's horse- he's not the sharpest tool in the shed, and I don't work with him regularly, so using my voice and posture doesn't work as well with him because he's not used to it. So cookies and carrots it is. But before I break out the cookies, I prefer to use my voice and stance, and a scratch if they like it, because I feel that's a better reward.
-edit- oh, and if I'm on our property, I let them munch on grass while I hose them down (if they got sweaty). Although I don't think eating of any kind is the best reward, it's kind of more incentive. Plus, it's a good treat for them (better than carrots) because I live in the desert so they don't have access to any grazing material. If that makes sense, lol.