My guess is because horses are prey animals and those big scarey animals are looking directly at them (like they might be lunch). God forbid they let out a mooooo or look out(!) here they come running!!!!
Just my guess.
Oh, something I remember hearing. Horses are very brave when things are moving away from them, not so brave when moving in their direction. Certainly seems true, judging by how "brave" Walka is with chasing, but not being chased.