Personally I think ny arena work is absolutely boring. If I don't have the trails, cliffs, rivers to cross and other challenges, what fun is riding a horse?
Make a decision if you really want to trail ride. There is no shame in not going if YOU don't enjoy the ride. There are too many things to do in life, Choose the ones you enjoy and avoid the others. I enjoy skiing, I even worked the Olympic Downhill races during the Salt Lake Olympics. But I don't enjoy going 80 mph on skiis, So I personally don't race. I still enjoy the sport, but from my own perspective.
If you really want to do trail rides, Then you need to learn how to deal with your fears. First of decide if you horse is safe for these endevers. You would never take a used car off a used car lot and go attempt to break a world speed record at Bonneville salt flat. With out first make some modifications to that car. Roll cage, safety harnesses, tires rated for higher speeds etc. Same with your horse. Evaluate if he will be safe to take out on a trail ride.
If you feel you can control your horse in the arena, thn it's not a big transition to controlling your horse ont he trail. We start colts in round pens for a reason. We want to control just how far they can run. But I only do 2-3-4 rides in a round pen before I head out for a trail ride. I just want to know that the colt will stop and give his head regardless of how excited. I can teach all the other controls on the trail. I don't enjoy a horse going bucking and crowhopping off thru a dense forest of trees. So I want to know that I can do a one rein stop if my horse gets spooked on the trail.
Next pick a proper trail to go learn on and choose some companions to go out with. Somebody that a calm horse that you can follow down the trail. First ride, I'd probably avoid trails with water crossings, steep up or down hill, cliffy areas. Ride someplace where you are not feeling threatened. I do like trails that are narrow and have lots of brush along side the trails. These seem to help focus the young horses attention on following the lead horse. Like the walls of a round pen they seem to constrain the young horse to stay on the trail.
My mares are so herd bound, that they would never stray away from the rest of the group. If I lead a group, and I put beginners on those mares, I know they will follow me anywhere. As suggested maybe you should go for a ride with somebody that has a babysitting horse.
The girl in the Blue Shirt is from Germany, A foreign exchange student that came and lived with us for 6 weeks. She had never been on horse in her life, We put her on the mare and she followed us up and over a pretty good sized mountain and had a wonderful story to tell her parents when she got home