There is nothing wrong with starting him over single poles and pole sets. That is not going to hurt him at 3. 4 is a good age to start over cross rails and jumps under 2 feet. But there's nothing wrong with poles. Around 3 and a half maybe start free jumping him over small jumps. I wouldn't do it everyday, maybe just once a week and not big, but that won't hurt him. As much as my 3 year old gallops, bucks, and rears in the pasture playing, some small jumps are not going to hurt them.