I weight 145lbs about 5'9" and ride a 1250lb QH and a 800lb grade welsh without any problems.
The QH has a bigger dislike for heavier riders in comparison for his size, he thinks that I'm the right size and anything much bigger is just asking too much. He is much more forgiving of beginner riders (on property) if they are lighter and much less forgiving of a heavier rider if they are unbalanced.
The pony can and has gone all day with me on her back with no lagging or issues keeping up. Not just walking around either, trotting, cantering, galloping, running through water, up and down hills, and racing a couple of QHs. She's a willing little creature and very honest, but sensitive enough that I have no doubt she'd give me a lot of hell if I was too much for her.
It depends on the horse, like so many things in the horse world. A rider must learn to read their particular animal and determine for themselves what they are capable of doing. For the most part I think "heavier" riders are far more aware of their horse's limitations than lighter riders. I know when I ride the pony I'm watching for signs of me asking too much more than when I ride the QH.