What type of horse do you have? How old is he? What is his level of training?
I have a TB mare that starts at the canter because she's been standing in the stall and she's just full of energy. After we lounge her and she makes a dozen laps in the roundpen she calms down and is ready to listen and go to work. For me becasue I know my mare is just young and full of herself I allow this.
Maybe you could put him out to run in a pasture or paddock before you try to work him? IF not just choose a time to work him that nobody else is there or block the gate for the first 10 minutes you work him so that when he's acting wild (which I don't think it that unusual) nobody gets in the way.
Do you have a round pen where you could free lounge where other riders are not at risk by your horses behavior?
Also I normally don't start the lounge with side reins. We allow the horse to warm up freely then put on side reigns. That way the horse is allowed to stretch and warm up before being asked for more taxing work.