I don't think what the OP did was wrong... its a shame the deer was wasted though.
It's just my opinion (and obviously it might be very different to others on here - and where I come from few people hunt and not many people have guns) but I would not want anyone on my property with out my permission EVER. I would not want shot animals on my property. At the very least I'd expect people who are planning to hunt around my property to come talk to me about it first, let me know what they were doing, where and when.
I don't know what sitatuion the OP is in, but many people know their neighbours and community pretty well. Perhaps she is making a sweeping statement about this neighbour - but maybe she isn't.
If I found a dead animal on my land and suspected someone had shot it on my land then I would call the relevant authorities. They could tell me if it didn't happen, or if it was legal etc. That is what the authorities are for - they serve the public. The man could have come over and apologised for the deer coming onto the land when he saw the family was upset, he could have offered to remove it, or something. He didn't though, he bailed. If he had stayed and explained the situation then the police may not have been called. He could have gone round to all his neighbours and asked for permission to go onto their lands - but he didn't.