I milk 14 head (at the moment, it fluctuates between 5 and 20) every morning and night. Milking isn't really my source of income, as they're show cattle so they're raised to be sold to breeders. Anyway, since most of the cattle are worth money, I'm really crazy about who can milk and take care of them, so yes, I definitely agree that it is hard to get away. I also own 2 goats, dogs, 4 horses, and a few beef cattle.
Another thing I find difficult, is bad things just seem to happen all together. Sometimes I feel in over my head and having a good part of your income depending on living animals that may or may not always be consistent can be a struggle.
Its also hard to do chores with cranky kids when my boyfriend cannot be there to keep them entertained inside. Actually, it gets difficult with the kids (2 and 5) and chores on a good day I guess. They listen well, but they are still new to having animals, so I tend to worry.