A lot depends on the temperament of the colt. A while back I had a 2 ur old Dartmoor colt that ran with another 3 yr old colt. The 2 yr old was a beggar and knew there's was more to life than just gazing and growing. The other you would never have known he was intact.
I have handled a lot of stud horses. A couple I handled were difficult in their temperaments, mouthy, pushy and one thought in mind and it wasn't anything to do with me to being ridden.
I was consistent in my handling of them. I insisted on total manners. I also worked them (riding them daily) a lot. This wasn't just in an arena but out and about. One, a TB was one of my best Hunt horses, he would stand quietly holding a gate open, I could lead other horses from him, including mares, and he would stand with horses all around him. Many didn't realise he was a stallion.
I always kept a stud bridle for when he was covering mares, different to his riding bridle. When I first started handling him to his mares he was so excited he would rear and buck going to them, thismwas firmly and fairly corrected so by the end of his first season he was walking politely to the mare.
On maturity of most males a lot of their brain cells migrate south, it is up to the handler to keep their mind engaged and occupied then they flourish.