I'm 5'8" 140 lbs with a 36" inseam. All of my riding horses are between 13.3 and 15.3. I was working a paso for my trainer when I was a teenager and could link my ankles under her heart girth. We looked funny, but I was only about 115 lbs at the time and she had no problems carrying me.
My 13.3 pony still looks pretty funny with me on her back, but she is my favorite for cow work. My husband is 6'2", and he has ridden a 14.1 draft cross mare for years now, she had no problems with him on her back, she's a stout little thing, but they make a funny looking pair, especially considering she is a black/white paint with a "fairy tale" white mane. I recently got him a 15.3 sabino gelding, and they are much easier on the eyes together!
For me to actually look normal on a horse, they have to be really fat or really tall. Drafts and warmbloods look alright, but they aren't exactly the best for what we do.