I agree with the others, he is a big strong guy and has learned he can intimidate the beginners. Is he yours or are you just one of his riders? If he was mine I would keep the beginners off him until I regained his attention. If you are one of his riders, you may have to spend more time pre-ride getting his attention.
When I find my three year old isn't paying attention, I bit her up on the lunge and have her work on transitions until she is ready to listen. Then I get on and do slow work, bends, transitions, pivots, moving her body. If she is good, we end our ride on a good note. If she gets naughty, I get off and she goes right back to work on the lunge, bitted up so she doesn't get to play at the end of the line.
My advice is make sure to find the time to get on him more often, or lunge him the day before you ask for work under saddle...
There is no passion to be found playing small - in settling for a life that is less than the one you are capable of living. - Nelson Mandela