Trust me, its fine. I know people in the 300's who ride stocky QH's and I've even seen one ride a VERY stocky Welsh. Yeah. Ponies are frikin' strong.
Anyway, Look for something with a longer back and big bones. And taller is probably good too, since you're definately pretty tall.
Get a lightweight saddle too. I used to help my father guide hunting trips and I had this idiot who I was in charge of teaching (Shwing him how to use a gun and saddle his horse and stuff. Eh.) and he went out and bought the HEAVIEST frikkin' saddle he could. An ancient, creaky, slippery leather saddle that I swear was over 150 lbs. And this was an overweight, disgusting looking city slicker who nearly shot my foot because he pointed a LOADED rifle at me after I told him not to. -.-
His horse was a QH, but he was more of a runner than a stocky pack horse. And when you're riding miles a day through rough terrian... Oh, hell
no. I saved that horses tail so many times by saying he was lame in his hind end after heating up his leg with my camp oven on low. (Too low to hurt. I wrapped the waffle plate in a towel and held it to his leg for 60 seconds and it was hardly hot but he believed me. Idiot didn't even notice.
Anyway, that's my story on why you SHOULDN't be like him, even though you don't sound anywhere near overweight at all. Just gotta make sure to get the right horse.