We have had very little trouble on trails, just annoyances.
1. Some hikers do seem to feel that the trails belong to them sometimes, but most offer to step out off the trail if it is narrow and steep and would be difficult for the horses to do. And offer compliments about the horses.
2. Some dogs will charge at our horses, but that is why its great to have our dog-friendly dogs with us. They are the meet and greet brigade! I do keep a can of hornet spray hanging on my saddle if I am in a place where dogs have bothered me in the past.
3. ATV's on trails where they are not allowed. But again, 90% of ATV riders we run into are courteous and kind, offering to let us go by, worried about our horses. Of course, my horses don't spook at ATV's. They are more likely to salivate as they approach, looking for a leaf of alfalfa!
4. Kids partying in campouts. Last summer, we rode into a remote area called Seven Lakes Basin. We encountered a group of teens at Cliff Lake that I think threw rocks at us while we floated on the lake.
Haven't seen any pot gardens in years, but that may be due to the fact that cows, elk and deer are very fond of hemp for dinner.