I actually feed mine cheapo oatmeal cookies as treats. I break them up, so the horse thinks they're getting more than they actually are.
I had a question about oranges when I first got my gelding, Casper. He loves anything orange, so don't feel bad if you don't know what a horse can and can't eat. Sometimes it's hard to figure out!