There are odd exception to the rule though. For example my Clydesdale is a nightmare to manage. He had an eye removed at 3 so has trouble regulating his temp (he over heats very easily). So you can't rug him in the winter because even in a rain sheet he sweats up. You can't stable him because his sinuses go mental and he stresses out in the stable causing to loose condition no matter what you feed him.
So in the winter he gets a bit skinny, mainly top line condition. If you were to look at him you would think gosh he' a bit skinny, but look below his hips and spine and he has a great covering over his ribs, he isn't cold, he is happy and active etc etc and of course in the summer he practically rolls round the field! Lol
But someone who doesn't know him would think I was neglecting him. I have a very close relationship with my vet and have him looked at the second his condition changes as there is always the worry that the cancer he had as a youngster may come back.
My point is that not all people neglect their animals and it is just the way that that particular animal is. It is a hard one to call if you don't know the people involved. Sadly all too often neglect is one of the main factors