How strange that he behaves all right when you ride him but on the ground he's so different. His behavior with the other mares that you described, I wonder if he's a cryptorchid.
Does he behave the same way with other people? If so, do you notice whether he behaves that way with men or not, or with other women or not?
Presuming he does this to everyone, I have what might be a weird idea but if you think it's worth a try...every time he turns round when you're close to him to try to bite, instead of reacting with dominance and aggression in response (I know the usual way is usually appropriate but it sounds like in this case he needs a different tactic), I would have a pocket full of treat weapons ready LOL and if you can be quick and on your toes every time he pulls his nasty tough guy act, move into him with your body right next to his neck and semi-under his head so he can't get at you, stay next to his skin and give him lovin, rubbing his face here and there. In other words, doing the exact opposite of what he's been expecting from you. Throw in an occasional treat, and basically ignore his antics but give him the exact opposite behavior he's trying to instigate with you ;)
Aggression is almost always fear-based, so being next to his skin is horse language for "I am here and you are safe." Move in, not away. Bring yourself close to him, don't push him away or threaten and such.
Watch, it'll confuse him for awhile, and he'll no doubt test you, but if you stay consistent and don't give him what he's learned to expect, I think it'll work. Eventually he may mellow out with the mares too.
I know it sounds entirely crazy, but I've had it work, so hopefully it'll work for you too. You just gotta be quick, always quick, and consistent.
As for any future reprimands, if he gets over the biting and such, it may be best to simply ignore him instead of get cross with him if he regresses at any time.