As people have mentioned teaching your horse to listen and be more sensitive to your leg will help out a lot! When you tell your horse to change gaits, it should be like pushing a button, not driving him forward until he falls into the canter. To get him to move off your leg a little better think "touch, squeeze, kick". Ask him for an upward transition with just a touch with your leg, if he ignores you squeeze, if he still ignores you give him a kick and maybe even a tap with your crop behind your leg. (don't pause too long inbetween) You need to get his attention! Then downward transition for a bit, then ask for an upward transition again doing the same thing. After a while, he'll start listening to your leg and waiting for the touch.
Also it might help to work on your timing, thinking about your horse's footfalls, and get it as precise as possible. For example, from the canter to trot you want to ask for the trot during the first beat of the canter (when his outside hind leg touches the ground). Because the 2nd beat of the canter is his inside hind/outside front.... which just happens to be the same footfall for a trot! So if you ask for a trot during the 1st beat of the canter, he'll do the second beat and automatically be trotting. Very smooth. Does that make sense?! I'm not the best at explaining this... :roll:
ALSO, make sure that your horse is balanced! His trot from a canter shouldn't be any faster or bigger then any other time he trots. If he goes from a canter to a frenzied trot (which would cause you to bounce around a bit) he needs some balancing work. Not sure if this is happening, but just thought I'd mention it.