Hi & welcome,
See you've got a fair few responses already, sure to repeat what many have said, but...
Originally Posted by asteriskmonkey View Post
One of my pony's has started to become very aggressive lately and I know male pony's from what I understand like to be a bit domineering and try to be boss.
Nope, unless he's a stallion(in which testosterone has some effect), gelded male horses aren't likely to be more 'domineering'. And even if he were a stallion, you can still teach him to have 'manners'. ALL horses need leadership, or else, depending on personality, they can become insecure, &/or try to assume that leadership themselves. Some horses can be bossy & domineering personalities about it too. Even previously 'well mannered' horses can become 'bossy' with someone who is not an effective leader, or doesn't consistently teach/expect good manners.
I was working repairing fencing in a field and the pony started aggressively biting and after trying to push him aside several times,
he then decided rearing was something he'd like to try out on me.
That just sounds like he thought it was a game. That he thought your pushing him aside was just an invite for more 'rough housing'. If you're going to use punishment, one important factor is that it needs to be punishing - unpleasant enough - to cause the horse not to want to be subjected to it again. While I personally use very little punishment & rarely 'strong', in this sort of situation, where the behaviour is not just annoying but dangerous, I'd be carrying a big stick(to also allow me to use it at a safe distance) and ensuring the pony was left quite certain this behaviour caused very undesirable consequences!
I;m worried as I have 2 small children, I bought this guy as a long investment into making him somewhat of a petting pony to leisurely enjoy his days without much stress. (hes 5 i think)
Of course it can be 'corrected'. It sounds like you have no experience with horses, and you've bought a young, untrained horse. Considering this, not to mention your young kids, I'd strongly consider moving him on and finding an older, well trained, kid friendly pony, *and finding someone experienced to come advise/give you lessons.
If you are going to keep this pony, firstly, keep your kids away(from any of them, if you don't know how to control/teach them effectively), and I'd be a) sending him to a trainer for a couple of months, and b) taking lessons with that trainer in the meantime & when he comes home. And c) keeping your kids away until he is reliable AND you are confidently effective with him too.