JMO the horse will need 2x as long as they have had off in order to re-condition. So 2 weeks off = 1 month of conditioning.
It is very difficult to let a horse down and start him back up as the joints lose condition, the cartilage can become brittle, the bones lose density, etc.. as he is let off, and it takes lots of time to recondition him to a point where his body is functional. He will remember the training, and he will want to come back into it, but it is very easy to overdo it and cause irreparable damage to the musco-skeletal system of the horse.
After competition season is over, I do like to give the horses a break with lighter training, lots of hacking and conditioning work, and less arena work. For about a month or two in September/October I will work almost exclusively in the field and on the x-country course with my dressage horses to get their fitness UP for the training season, without the stress of arena work, as I do a lot of arena work in winter as there is no where else really to ride. They appreciate the break from arena work and come into the winter season as refreshed as if they have had that time in a field, but with stronger bodies from the conditioning work. Then I can train in the winter, come out in February for the spring season, and take a similar break for a week or two in May to condition them again for the summer season. Then repeat. So the horses are not always in the arena, but they are always in good condition. I find I have almost no colic, ulcers or functional lamenesses (horses will always find something to cut themselves on or kick something or slip on ice though!) and they are in an intense program. This along with good feed, weight management and a good care regime (every day turnout, big boxes, consistent feeding 4-5x a day, etc..) so the whole program.
tl;dr - No I would not let a horse down for any reason other than a lameness, vary the work schedule on the off season to keep the mind fresh and the body conditioned.
They say money doesn't buy happiness -- well happiness doesn't buy horses!