I guess it depends on if I just bought the horse or if I have it on trial.
If I have the horse on trial, there isn't really much to loose, so I usually work with them right away. That way I can see them at their worst- in a new location with a new rider. Might as well find out how they handle stress.
But before I even bring a new horse home I try it out at the owner's place. Then I know if it is barn sour and how it behaves in it's comfort zone.
The exception was my mare. I tried her out at the owner's place and then brought her home and let her settle in for the day because in buying her and taking her home she was suddenly weaned from her previous foal. It wouldn't be fair to just take her home and ride her that same day. I would expect her to act up and look for her baby.
I was prepared to give her several days to settle in before riding because 1) I figured it would take that long for her to get over loosing the foal and 2) I had already bought the horse so there was no rush to find out what her behavior would be- I was already committed. But alas the next day she was acting calm and no longer calling for the foal, so I called up a friend to ride with me and we did the first ride on the trail with another horse. It worked out great.
Actually, that is what I would recommend. You can try the horse out in the round pen or whatever, but on your first ride with your new horse outside a pen or arena, try to get someone to go with you for safety's sake. Then if you have trouble you won't be all by yourself.
But I really don't give them much time to settle in. I WANT to know what they are like when stressed. I would rather find that out right away then have it surprise me later.
If they handle the first ride or two well, it gives you confidence and everything after that should just get better and better.