Failbhe mine doesn't do that, it zaps if they stay in the beep area for too long as well as if they go into the zap area.
She has definitely calmed down a lot now that she's a little older, she's obviously not testing it as much as she used to. But as soon as that battery is dead she's still running out where she's not supposed to be so I don't think I can ever just turn it off.
In our case, I think a lot of it has to do with the fact that when we first moved here she could run wherever she wanted, and then we put up a fence. I think if she'd been taught to respect the fence right from the start it would have been more effective.