The more turnout time, and the more social interaction, the better. It is neither physically nor mentally healthy for a horse to be cooped up in a stall. Quite honestly, they do best with a big herd and many acres to roam, 24/7. They live longer, are healthier and happier, and develop far fewer vices.
That said, if your horse has always been solitary, he may not have developed the necessary social skills, which could put him or his herdmates in danger in a group setting.