I have often thought too, that blankets go back to the days when saddle makers used REAL shearling fleece under the saddles, not the cheap synthetic stuff they use now days. If you have a nice underside of real fleece and the saddle fits well, you can get away with just a blanket.
If you have the synthetic fleece, which just about all production line saddles have nowadays, then it gets flat as a pancake right away and you need more padding, especially if it is a less than perfect fit.
I do have a saddle that fits so well (has never caused a white hair even with 5 hours trail rides several days a week) that I will sometimes use two single layer saddle blankets instead of a pad because I like the way they look. They are the nice, wool Maytexs, the thick ones. But still, I will almost always use a pad and a blanket together because I figure it gives the horse the most protection.