It would depend on his price. If he's low enough that he could be resold for most of what you pay, as a trail horse, then I might chance it. If he's high enough that you couldn't afford to replace him without selling him high, I would pass.
Starting jumping at 4 would not bother me, he's probably only been started over cavaletti and 18" - 2 ft. obstacles, and probably not many of those. To me, that's just training fences, if he were JUMPING say over 1.5 meters, that's another story.