I'm going to take a guess that your horse's flatwork needs a little work. Before I really start a horse's jumping training they need to be able to have 3 solid gaits and know how to move off my leg. So a leg cue to them doesn't just mean "go faster" it can also mean "move over". (leg yields particularly but also turn on the forehand, shoulder/haunches in, etc) That way, when a horse starts to drift or try to avoid the fence I can fix it. You need to be able to make a "chute" for your horse using both reins and both legs so they have no option to go forward. But unless your horse can move away from your leg there isn't much you can do. I'm also going to guess that he is knocking down poles, not just because he can't find a distance (at the beginning of his training this is your job, not his) but also because he doesn't know what to do with his body. I suggest lots and lots of gymnastics! It's the best way to teach a horse what to do with his body while he jumps and eventually find his own distances.