However, many people don't leave this patch. I have done total clips with no problems at all. If a horse is sensitive at first, they will get used to it. Then, next time, you might leave a saddle patch.
Personally, I wouldn't worry and that person has no business criticizing you.