By throwing the contact away suddenly, you can make some horses nervous. You are essentially leaving them on their own with little direction.
Keep your contact, make sure you have a good working trot that is off the forehand, in front of your leg and with your seat. On a circle, leg yield out off your inside leg for a few strides, then cue for canter.
Some horses will canter off the inside leg, others off the outside. It depends on how they have been trained.
Also remember that your seat has a huge part in the canter cue. On my own horse, I find that I get the best strike off to weight my outside seat bone, 'lift' my inside seat bone forwards, and ask with both the inside leg on the girth, and outside leg slightly back.
If you start shoving your outside leg back as far as it will go, you'll tip your upper body forward, pinch your knees, block your seat and piss the horse off by kicking it in the flank. Totally non-productive ;)