Depends on the situation.
If we are just meandering around on the trails and through the fields, I allow my horses to snatch a nibble as long as they keep their feet moving. I figure if I am enjoying myself it's only fair that my companion gets to enjoy as well.
However, if we are working on patterns or any type of schooling in the field, the mood is different and they learn to concentrate on the job at hand. Once the lesson is over, I'll sometimes stop and dismount and let them nibble a bit as a little reward for a job well done.
I'm all for rewards!