Yes, do work him a little extra his stiff way, but don't neglect the good side entirely. Be sure to help him out as much as you can (sit straight and don't lean yourself, and remind him with your inside rein and leg to stay even, support with your outside leg and rein, sit a little extra on your outside seat bone). Try doing spirals at the walk and trot (and canter, once those are mastered), starting with something like a 20 meter circle, spiraling down to something very small, and think about making it "pretty." A visualization that I use is to pretend that my inside leg is a fencepost, and if the horse is bending well he won't lean any more than a plumb fence post, if that makes sense.
He might never be perfectly even, most (I'd even venture to say all) horses aren't. Like people, they tend to have a dominant side, right or left handed. You might look into an equine massage therapist or chiropractor. If he's really muscle stiff or out of alignment it can be harder for him to bend well.
A stubborn horse walks behind you, an impatient one in front of you, but a noble companion walks beside you ~ Unknown