Ok so more info. The more warmed up he gets the better he gets, like by the fifth or sixth time jumping, just in general not a specific jump like any jumps at all, he is doing better and will even do it at a canter. When I do it, I don't know if this is right but its how I do it, feedback appreciated on this too, is I'll get him warmed up with flatwork then start off trotting single fences. Once he seems warmed up over fences and is finding his distance, I'll try a few cantered and see how he does with those. Once he's got those I might do a small line. And then by now he's probably tired so I'll cool him down.
So I am consious of the staying with him until the jump, but the one time I tried really hard, I was connected to his mouth, and I was firm with my legs, the only thing that happened was I got severely left behind when he jumped from a stride away.
So even if it is my fault, what can I do to make this better?