I really think my opinion depends on the situation.
I had a friend that used to let me ride their horse, before I got mine. He was an 8 year old gelding, I'm still not quite sure what breed he was. He was a GIANT horse, between 16-17 hands if my guesstimation is correct. Well anyways, two years ago, I used to go out and ride him about every 2 weeks or when I had time. He was kept in a small pen in their backyard, a bit more than 1/2 of an acre. He was in this pen all year round, so naturally, it was eaten down to dirt. The fences were chain link, but he had leaned on them enough so they weren't really fences anymore, but they did have an electric fence installed. He well taken care of for the most part, but sometimes his hooves where longer they hey really should've been. I wouldn't say he was happy, I would say he was melancholy. He wasn't being mistreated, but looking back (now that I know more about horses) I can tell he was bored. Most horses, when they hit the age of 5-7 years old, need a job. They need something to do, some kind of challenge to keep them busy. He was well trained I'm sure, at one point, but I think that sitting in that pen made him forget most of it. He really just needed attention and some stimulation.
My point with this story is that it really depends on the situation. In this case, I didn't necessarily think that them having this pasture pet was a bad thing, because they did take good care of him. I just think he was very bored, and needed to have something to do. It just bugs me that in that situation, that horse could be so much more than what he was. :/