Its not just physical soundness that must be taken into account though. Two is pretty young for just about anything, and they don't seem to retain much more than the very basics at two anyways.
Drafts and heavy pony breeds are notorious also, for really not being done growing until they're 5, 6, and sometimes even 7. Therefore a young draft's body is still in the very fragile stages of development as two and three year olds. As Loosie said, if you're light and have a light, well fitting saddle (bareback is not good for youngster. I never ride them without a saddle until 5+) hopping up to give him a quick starter course in 'walk' 'woah' 'left' and 'right', it probably won't hurt, but you can do just as much ground driving, both at a walk and a trot at that age, without worrying too much about stressing their bones.
Either way, let your horse be a baby while he is a baby. There is so much besides riding that you can do with one, two, and three year olds- so don't feel like not being able to ride your horse let means you're missing out!