I hate pushing a product, but I found Clinton Anderson's dvd series "Handling Foals, Weanlings and Yearlings" to be extremely helpful. When I got my weanling, he was pretty much where yours is training-wise and I was wondering the same thing "what the heck am I going to do with him...?"
Needless to say, I've found a LOT to do. Desensitizing is very formulaic and easy once you learn the basic process. What the videos have helped me with most is sensitizing him to pressure and gaining control of his feet. He's always been very good-natured and calm, but he was also slow to react when I ask him to move and downright pushy sometimes.
Now, I can put a finger on his nose, say "back" and he will...which is great when you enter the stall with an armful of hay. If he gets crooked in the crossties, I can basically look hard at his butt, shake my finger, and he'll step over straight in back. He no longer crowds me in his stall or at the gate of the paddock. Those are just a couple of examples which make life a little easier, but having control of the feet becomes invaluable when you take a colt into new situations.
It could be going to a scary new place, like a show barn, it could be just walking past the flirty mare in heat a few stalls down, but having those ingrained ground work exercises gives you and your horse a safety zone to fall back to. He'll have a lot more respect for you as the leader who keeps him safe, and you'll have tried and true methods to deal with anything he might do, from bolting to rearing or striking.
I didn't mean to write so much, but I really sympathize with you. Wikke is the first horse I've owned this young; I had a lot of the same thoughts before he came home with me :P