My mare used to do this when I first got her. Every time she did that, I'd push her sideways. Not in a punishing way, but a calm methodical lateral movement until she accepted that leg pressure, then I'd move on. You have to make sure your aids are remaining clear to him. Try not to let them all mush together so that he's desperately trying to find the answer.
I would not use a martingale or any such type of equipment on this type horse. His head is up because his back is down. You try it. Go get on your hands and knees, and make your back slump like a swayback. Now try and drop your head and notice where you're straining. Now lift your back, and notice how easily your head can drop in comfort. You cannot force a horse's head down if his back isn't there to support him, it will only cause him discomfort and make him more tense.
Go back to some ground-work. Do some in-hand work to teach him he can be forward, head down and his back and belly engaged.
Put your bridle on him and go out to your riding area. Put the reins over his head, grab a whip. Place your hand just below the bit on the rein, and your other hand will hold the outside rein and your whip. WHen you go to use your outside rein you will pull down. Not up, back, out, etc. What you are asking to begin with is the inside rein pulls down for him to lower his head. Once he does, release. Do this until you think he's got the basic idea, then you can start to add in the walk. Ask for his head down, then gently touch him with the whip on his hind-quarters, and walk towards him. You want him to bend around you and walk with some lateral intention. Don't try and hold his head down, just keep fixing it. Askf or it down, ask him to walk, ask for it down, ask for a walk. He will probably stop alot, it's fine just keep asking his head down and then something forward. What you are doing is pushing the hind end forwards, while encouraging him to lower his head. If he walks forward with a lower and softer neck be invisible in your reins so he knows that's the area you want him in.
Basically, he doesn't know how to lift his back and the muscle under his neck is stuck. Lateral stuff will helps this the most. Stay away from draw reins, tie downs, martingales, etc.
If you like this idea and want more info, let me know. There's alot to it. :)
In riding, a horse's energy is like a river- guided by the banks but not stopped by them.