While if you were going to buy him a horse, I would select one a little larger, I think you guys would be okay for leisurely trail rides. Just give it a go and see how the horse does.
My first horse was a 14.3 Arabian, and I weighed about 225, and he carried me fine. But I do feel more comfortable on a heavier boned horse. Not that the Arabians can't carry me fine, but I just feel less self-concious on a bigger horse.
Actually, my current horse is only around 14.3, and I've lost a few pounds (yay!) but he is a Mustang and very stocky for his height.
But for a leisurely trail ride, I would say give it a try and see if they both are comfortable.