No worries, I totally feel the same sometimes.
Lacey's 14.1hh and 950lbs when she's "perfect" (since she's usually fat, she's more like 1100lbs on a regular basis). I'm 5ft 8inches and 145lbs so while I'm not too heavy for her, I'm really a bit too tall for her. Thankfully, she has a "well-sprung" ribcage so my feet don't hang down as much as they otherwise might and I feel pretty secure on her back. My biggest complaint about Lacey is that the widest part of her belly is pretty high up, which means that it's very difficult for me to effectively use my calves on her. I'm always forced to put my heels up and use them to cue her, it's kinda a pain.
For funniness factor, here's a picture of me on a 13.1hh Quarter Pony mare who probably weighs 1000lbs on a skinny day (excuse my face, I have no idea what I was looking at! haha):
The funny thing is, my legs look perfect on her but my torso looks huge! Trust me, my legs aren't actually THAT short, that mare is just so round!
Here's a picture of me on 14.1hh Lacey, for comparison (ignore my legs...I was possibly adjusting my stirrups? haha). That mare tied to the fence in the back left of this photo is the one above. Lacey's a lot more leggy than she is, however, the two of them fit the exact same saddle!:
I do feel more "in-balance" on the shorter mare, I think due to her bigger body (she's my favorite camp horse, I try to ride her whenever possible! She and Lacey hate each other. haha!). It's weird.