I totally agree with that. Groundwork first, but using LEG to back your horse. My goal on a finished horse is to back with only leg and no use of hand at all, so to get there, you start with hand and leg, but don't forget the leg!!
Legs should be the ultimate goal. I start on the ground with voice cue ("back") and tapping on the chest/front of shoulder. Then from the saddle, Sit back feet forward tap with your feet as far forward as you can reach. This is really exaggerated till your horse gets it down. So your tapping almost where you would be tapping from the ground, does that make sense?
Gradually you won't have to go so far forward and just wiggle your feet and the horse should recognize the cue. Ultimate goal again will just be legs forward, seat back, without using any bit at all.
I'm not a horse trainer, this is just the way I do it. If its wrong feel free to correct me.