I feed ditch hay.. But I also help bale the hay I'm feeding and know exactly where its coming from. Occasionally we'll find the odd bit of garbage here and there, but its never been enough that I was terribly concerned about it.
It really doesn't have much nutritional value, as someone pointed out, but my horses are given good mixed hay two times a day and since I keep my horses on a dry lot, they have free choice ditch hay (usually) all the time.
You should always be checking any hay you get, so of course, check this as well. As MacabreMikolaj
pointed out, cropland is all fairly close, so I wouldn't over-think what pesticides and such would be on this compared to hay grown in a field. Its probably close to the same.
As for knowing the difference.. Ask out-right to see where the hay was baled. There's always the risk that he won't be honest, but hopefully he would be.