(Indie's mom was full walker, and she really looks the part, so I'll go ahead and throw in what I know)
I agree with the above posters that they are commonly narrow (and a little dainty) built, but shouldn't really be "skinny." Most walkers I've seen have been on the narrow side, I know for Rackers this was a bred trait by plantation owners so that when riding the horses through the fields it would be easier to ride between the crops instead of over them.
Now your little walker may LOOK skinny compared to a bigger boned QH (oh boy... it sure does look funny seeing my little Racker in a herd of big boned Quarter Horses
), and when they are super fit it wouldn't be uncommon for the first few ribs to be more easily felt and seen, but your horse shouldn't be "skinny" as in you can see most or all of the ribs, or the butt is more sunken in than it should be.
Hope that helps :)