I'm pretty new to endurance, so definitely not an expert, but from an eventing background and working in the veterinary field, I am much more concerned about tendon/ligament strength than I am with cardio fitness. My horse will tell me loud and clear if I am working her harder than she can handle from a cardio perspective, however, I have no way of knowing for sure that her legs are fit and strong enough to handle the stress of the workload... therefore, I condition more slowly and conservatively, just in case ...
I've been through a suspensory ligament rehab with a previous horse... not something I ever want to have happen again.
Better safe than sorry, IMO. I don't mind taking longer to get there if it means my horse has a better chance of staying sound longer.
My second-year horse I plan to do about twice as much with. If all goes well, we're really going to hit the endurance trail next year and load the miles on!
However, most people just looking to get into "endurance" can do so within a few months if they take it easy and enter LD's (or even 50's), give the horse enough time off and space out their rides, and don't turn it into a race.
The background plays a huge role as well - Just as endurance horses conditioned in the past come back to being race-ready more quickly than new horses, I would think it would be a much shorter "conditioning" period for an eventer!