Barrel, be careful with potatoes. Horses can't digest them properly, regardless of what your gelding tells you!
Pretty much any fruit can be given to horses, but you should try and stay away from processed foods as much as possible.
As an occasional
treat bread, cookies, and even dry cereal are fine. Horses also like uncooked pasta, but again, limit their intake.
When I mean occasional, I mean every 6 months or so, definitely not as a regular treat or even once a week.