All horses have a lazy streak, it's just a matter of finding it, lol.
I would try doing some "cruising." Ask for forward motion with a gentle squeeze. If that doesn't work, cluck, then spank with a crop, string, etc. until he moves at the gait and speed you want. Once he is going, completely remove the cue. Trust him to move forward at the gait you have asked for without micromanagment from the saddle. Don't steer, just let him truck around the arena or pasture wherever he feels like. When he breaks gait downward, let him take a couple of strides at the new gait, then use the same ask/suggest/encourage progression to put him back into the intended gait. If he speeds up, bend him to stop and soften his face, then ask for a trot again. This exercise makes him responsible for his feet, and shows him that walking when you had asked him to trot is wrong, and not to walk until you ask. Make sure that, when you do cue him to slow down, you don't just let him slow himself. Make the downshift your idea. You can lunge or roundpen him and apply a similar technique to make him responsible without a rider.
With the canter, I would lunge or roundpen him at the canter, then add a saddle and let him learn to balance that weight, then add a rider. It does sound like a balance issue to me. If you aren't extremely stable at the canter, I would enlist the help of a horsey friend who's balance issues won't interfere with the learning horse.
A stubborn horse walks behind you, an impatient one in front of you, but a noble companion walks beside you ~ Unknown