Hehehehe...he sounds like how my mare was when I got her last year at this time... I simply started over from the ground up; she was gentle on the ground, but had a lot of holes (nervous about feet, didn't know how to lunge, yield hip or shoulders, etc)...so I started as if she didn't know anything. I didn't really treat her any differently other horse, although some things I took more time on, but I really think the key to helping a horse overcome such issues is to essentially act as though they really aren't there; the more you acknowledge it (say he gets nervous about a bag making noise on a fence), by saying things like "good boy, or it's okay"...the longer he can take to get over those things. If you just continue to do what you are doing, say you were lunging, and let him work through it without interfering, he can figure out that it's really not a big deal, simply because he realizes you aren't reacting, so he realizes he doesn't have to either. I like alot of Clinton Anderson's exercises as well; like smacking a handy stick on the ground while walking away from him (horse is on the lead of course). You're not facing him, so he doesn't see you or the stick as a threat, and can figure out quickly that it's no big deal.
When he is consistent and calm on the ground, then start doing the same stuff from the saddle. He may come around in 2 weeks, or it may take him 2 months, maybe even longer. I was on my mare in 2 weeks, but while ground work has been easy once I earned her trust, saddle work has still taken a bit more 'work'; It's taken a bit longer for her to relax, and work quietly...the ground work was huge, though, in that atleast I knew I could bend her around in order to prevent any buck and bolt behavior. Once you do start getting on him, make sure you continue with the ground work as that will help you guage where he is mentally on a particular day.
"The ideal horseman has the courage of a lion, the patience of a saint, and the hands of a woman..."