The barn owner seemingly has a weak case and was probably bluffing when he first claimed the damages. He would not only have to prove your horse did the damages but also how exactly the figure was arrived at if he did sue you.
Now, he has a great case. You admitted - in writing - to the damages and agreed to the amount. You also agreed to pay back that amount.
If the barn owner has a lawyer and is going through the regular civil court system, it is not worth fighting it. You will pay more than the money owed for a lawyer.
Small claims court, however, will not cost you anything and is not something to be scared of. Worst case scenario is probably you spend some time in a courtroom and end up having to pay whatever it is you still owe.
Note that most lawyers will provide a free consultation and tell you if you "have a case." Even if you are being sued in small claims court, if it worth speaking to a lawyer to get some advice.
From what you wrote, the story sounds like this:
- History of late payments and asked to leave
- Barn owner claims damages
- You agree to damages and to pay back in full
- You look for way out of paying for damages and find a zoning technicality
- You decide to default on payment contract
You can probably get _something_ in small claims court. The zoning issue may be relevant, but I think it is a mistake to base your entire defense on it. For all you know, he may have obtained a variance and be allowed to legally board horses there.
Point out that you were pressured to sign. Did he threaten to sue you? Threaten to put a lien on your property? Take away your horse? Doesn't matter what he would actually be able to do... just what he threatened to do and if you believed it could happen.
Did you ever argue the extent or amount of damage done? Can you provide any witnesses to the condition of the property when you first started boarding there? While it is tough to argue your horse _didn't_ do the damage (since you already agreed he did) that argument is worth a shot - especially if you felt strong-armed into signing the agreement.
Finally - do you have homeowner's insurance? Check with your agent. Some or all of the horse "damage" may be covered. If so, they will handle everything and answer any lawsuit with their lawyers. Again, the "smoking gun" is the payment agreement you signed, which may still make you personally liable for the amount.