Yes, in most cases you're right. However there will always be the curious or young horse that tries the wrong thing. Some toxic plants may taste good too.
Wild horses have a choice as to what to eat & maybe some do eat the bad stuff & get sick or die & we'd never know.
Our domestic horses are usually on a limited area & often will eat anything green so it's best to remove the temptation.