I would only handle him daily for two or three weeks. He should have basic manners (like not walking all over you, letting you pick up his feet, brush all over him) by then.
After that I'd only handle him 3 or 4 days a week.
Take him on walks. Occasionally hand graze him. Braid his mane. Put curlers in his mane. Take him along while you weed a stretch of garden. Lots of plain of living that he can join in on. If you're wanting to show him in halter, get him used to all that. If you have another horse, pony him just a little.
But, the biggest risk is you getting bored with it all and pushing him beyond what a yearling should be doing. Mostly, being a horse and growing.