I've been reading and checking in with this thread for a while now, and decided to give my 2cents worth (worth less probably

).
While I agree with the majority here that taking your horse for a nice trail ride is good for their minds, if the rider is a ball of nerves, well it has the opposite effect for the horse.
9 years ago, when I brought the horses home, I gained the joy of having them here, but lost my riding partner. My daughter, who is a beautiful rider and has handled some pretty difficult horses with calm and firm gentleness, discovered that she disliked trail riding to the highest degree. She does not trust the unknown that can happen on the trails.
So , to please her mother, she went on a few trail rides, when Walka was being saddle trained, and was the absolute picture of misery. She was on a very steady, unflappable trail horse (T), and hated every second.
So I decided not to be selfish and let her off the hook. If we had an arena she would have kept riding with me, but we didn't have one.
So as Vida said earlier, and a few others, ride where you are enjoying yourself. For years T was ridden in an arena , only about 1 or 2 trail rides a year, and she was never sour. Partly because she was kept challenged and stimulated. I'm not a driller, if she got what we were doing, I moved onto something else or applied the lesson in a different way.
We have very little time, spend it the way you enjoy with your horse. If trail riding is something you think you might like to try, do it in small increments like Redhawk explained so well.
Bottom line, it should be enjoyable. In or out of the arena.
Good luck :)