Some horses are sensitive to the "bristley" feeling that clipped hair has under a saddle pad. Many people will leave a saddle patch to avoid this possibility
However, many people don't leave this patch. I have done total clips with no problems at all. If a horse is sensitive at first, they will get used to it. Then, next time, you might leave a saddle patch.
Personally, I wouldn't worry and that person has no business criticizing you.