3 years old is usually the ideal for most light breeds. Of the 3 horses I currently have one of them is a full draft so with her it would be a bit of a different case. I know I won't be getting on her until she is at least 3. Because they are so large, it really takes longer for them to fill in and their joint to close up more.
It really depends on the horse. Some lighter breeds also need more time to finish growing.