What you want to do with a horse influences what kind of horse you should choose. Regardless, there are horses out there who will meet your needs ... and they don't have to be big stocky draft horses that aren't particularly well suited for riding.
If you prefer huntseat, jumping, or dressage, find a warmblood (a draft cross). A good sized quarter horse would work too, as already mentioned.
If you are into pleasure riding or trail riding, Missouri Fox Trotters or Tennessee Walkers would be a sturdy fit. They're similar in size and build, but I personally like the personalities of the Fox Trotters better.
Any of those would work. Don't listen to anyone who would discourage you from riding. You just need to find the right horse for you!