Sometimes when you are in the business of training you have to weigh whether an owner like this will actually harm your business. I can't make that judgment for you.
If, as others have suggested, you think this person will mess the horse up once you get it going well and will blame you, you may want to have a conversation.
I generally say something bland like, "I don't think I can meet your needs as far as what your priorities for this horse are." One of those "it's not you, it's me" kind of breakups.
If they are at all bright, they may figure out that they are getting fired and either get belligerent, or beg you to continue training. If you have decided that you need to get this client out of your barn, don't weaken.
If you think they are simply ignorant and there is hope for them with their horse, have a completely different conversation.
I've only had to do this two times, so that's a plus. And both times the people blabbed about how I couldn't train their horse, but others came up and offered support for my decision. Which doesn't really matter, but I felt a little bad that I wasn't able to work with the people.