Hmmm, makes me want to ride with music while I'm exercising horses! Should be safe as long as you ride with just one earbud and tuck the wires, keep the iPod in an inside coat pocket. I don't see a problem with it.
I bet it would be fine, but I personally enjoy talking to my horses while I ride. I'm sure they could care less, I just like to praise them or make my silly "NAAAHHH" sounds when they do something stupid.