Height has little bearing on a horse's talent. I've been out jumped by ponies too many times to count.
Heart, soundness, athletic ability and willingness are probably the top things I look for. Granted your horse should fit you, but you don't need a giant.
My personal experience with horses over 17HH is that they break down quicker if under hard work. There is just too much weight and stress on them. Plus they tend to be less handy and need to be pulled together in SJ and Dressage.
If your horse says no, you either asked the wrong question or asked the question wrong
And God took a handful of southerly wind, blew His breath over it and created the horse