I'm 5'9 and I fit on my 14.1h mare just fine (even with western length stirrups, my feet aren't below her belly). It's all about how round or not a horse is. A wide, round horse is going to take up a lot more leg than a light boned TB.
I agree with Kitten_Val, it's all about your own preferences, your riding abilities, and your weight. Another thing that I think is important is the terrain you'll be covering. A smaller horse might have difficulty getting a big man up a steep hill but have no issues on the flat.