All sorts of things, babies are quite fun so long as you put the emphasis on manners. Feel free to follow my blog if you want to see my filly's progress.
I was away a lot at school this year, so I probably only got in about 4 weeks of solid "training" in her first 5 months (no longer than 5-10 minutes a session). Here are some videos of the things I worked on. Granted, you could probably make a lot more progress if you were able to work with your horse every single day, but balancing training with letting them grow up is good to have.
Giving to pressure at about 3 months old:
Working on some more natural horsemanship and advanced pressure and release at 4 months old:
And just some messing around. No use getting upset if they don't remember something or mess up.
With only four weeks of training, you can see the amount of progress you can make. Just remember it's important to set them up for the basics in life - leading, giving to pressure, mini join-up, hip disengagements, grooming, good behavior, and much, much more. I will certainly be able to make much more progress once I can get my hands on her consistently - and consistency is very good for babies with small attention spans.