Take it all in little steps. I am a goal setter...I would set a place in my mind on the trail (a significant tree, creek, etc) and say "we are going to ride to there and once we get there, we will turn back. If he spooks, I wont lose my nerve because it isn't that far and certainly he can handle going 1/2 mile into the woods. Accomplishing goals improves your confidence. Just make sure they are achievable goals.
Is he stabled by himself?