I give my horse treats. He gets the apple-oat flavored Dobbins treats, apples, black licorice and anything else that I feel like offering. Most things are reasonably safe if it's just a bite or two every so often, even if they are relatively high in sugar. Mine likes everything.
That said, there are no treats if he's not polite. The moment a horse starts crowing/pushing/or being an @$$ over treats, he's going to be chased off, and hard so that he learns that if he's not polite, he's not even going to be allowed anywhere near, let alone get anything tasty. With that, my boy has remained very polite and though I can tell he knows when I have a treat, and I know I want to give it to him, he stands quietly and waits until it is 'my idea' and then politely takes it. I try not to give it for free- I'll try to ask him for one of his 'tricks' first, but sometimes he gets them just because I want to give them, and so long as I'm 'rewarding' him for standing quietly and not being nosey, I don't feel too bad.
ETA: I had a bunch of stale granola bars in different flavors. My horse LOVED them. Made it really, really easy to 'catch' him out of pasture as he was practically falling all over himself to come in in hopes of finding one in his bucket when he got to his stall.