I think a big difference in which breeds certain riders prefer has to do with ability. Most english riders do hunters, jumpers, dressage, eventing. In all of those, a horse with long legs, a big stride, and good suspension is key; therefore, TB, warmbloods, etc. I think it is just that those particular breeds happen to be taller. As for your average cowboy, well, we want our horses to stay solidly grounded and able to drop down at a moments notice, suck back in front of a cow, and pull anything we put a rope on. That is just easier for shorter horses because they don't have so far to drop down to get in the ground. A taller horse is usually slower and clumsier in a sharp turn (cutting), and the added height puts more torque on their back when you rope something heavy.
Basically, most english disciplines need a horse that stays above
the ground, most western disciplines need a horse that stays in
the ground, hence the height and movement difference.
I believe that's probably why some people think they need a horse of a particular height to "look" right, it's just what they are used to and what is common in whatever sport they choose. I have ridden all sizes from 13hh to 18hh and I prefer horses just a little under 15, but the size that I am, I could probably get away with riding almost anything.
Barrel size has a lot to do with it too. TBs (I'm not sure about warmbloods since I have never seen one) generally have a narrower barrel than other breeds, so they need a taller horse to take up a long leg. However, QHs are generally really stocky and wide so they can be shorter and take up the same amount of leg.
Perfect example: My brother is 6'3" and the horse he is riding in this pic barely breaks 14hh. I still don't think he looks underhorsed.
(he's in the blue shirt in the middle of the pic)