Well they're so small.. maybe they remind him of dogs or he's scared of hurting them?
Never know with horses, they are quite the characters.
If he's scared around calves, work him till he focuses on you. As soon as he focuses, stop working.
He'll learn in time that they aren't anything to fuss over.
Even if he is a cutting horse, yes he needs to be able to read the calf but he still needs to pay attention to you.
I am not a cutter, but I do know how to ride a horse and know that losing their focus is what causes the spook. If they focus on you, there isn't any way they can.
"Strength is the ability to use a muscle without tension"