I love riding multiple horses! I have two riding horses now and an almost 2 yr old. I would love to have 3 riding horses. Heck, if I were rich I would have more.
I also love to ride friend's horses if they offer. It's a blast to ride different horses.
But at some point there is diminishing return, at least in a single rider household. Over a certain number a single person is probably not giving each horse as much attention as it could have. However, as others have stated, as long as they have food, water, shelter and health care, most horses don't complain if they get vacation time!
My reason for having more than one is that I try to ride every day in the summer, and I feel like if I have just one I am putting a lot of work on one horse. If I have two I can alternate and they will each be doing a reasonable amount of work that way, without getting overworked.