I don't see a problem with it, as long as the fabric is relatively unabrasive. Oh, as well you want to look at how well the saddle fits with the blanket under it; if it pads too much you could end up pinching the withers and creating a bad fit.
I like to, I think it's fun, lol. Back when I was in HS, we would buy fun colored western blankets that matched our horse's "colors" or the barn colors, and then use shaped fleece pads over that. We'd ride that way for fun shows or events. Of course, this was before there were so many color choice for English square pads .