I agree you are just forcing his head into position, which is not helping at all.
He also appears to be behind the bit, which is hard on their neck.
Ask and release...
Get a friend that knows how to tell when your horse has his head on the vertical and carrying himself properly...ask your horse to go on the bit, and the second he does, release....that is the reward for doing it.
Also what helped me horse from keeping his neck up high, was I'd ride in a rope halter with an extremely loose rein...so he'd put his head down, etc.....that helped strengthen his back legs and hind-quarters, and made him relax a bit more.
If you can ride with no contact at all and he will still keep his head up it's either there is some sort of pain issue OR his neck is naturally in an upright position