If you are not supporting him all the way to the jump and into it, he will be much more inclined to stop or run out, if he hasn't had proper jump training.
It's going to be difficult for you to teach him to jump smoothly without having been taught how yourself. I would make sure that you keep everything very small, and be aware of your body position over the jump. Practice a LOT of two-point and riding without stirrups on the flat to strengthen your core muscles and train your leg to be stable. And when you jump, make sure you do not jump ahead of him, allow him to jump up to meet you.
And be aware that you don't chuck the reins to him a step or two out. That's a common issue for newer jumpers. To the horse that's like saying, "I'm here for you, I'm here for you, I'm here for you...Nevermind. Do it yourself." If your horse is timid, that will make it worse. Just take it slow and steady, and he will gradually improve.
"Keep a leg on each side and your mind in the middle"