Sounds like a horse we have. This horse was ridden in draw reins, ridden with his nose to his knees. He's a very downhill built horse, which means his front end is lower than his hind. When he first came for training, my trainer was too freaked out by his way of going that she didnt even put him into trot. He just felt like he was running down a hill faster and faster and barreling out of control. lol.
Your horse sounds similiar, in that he puts his head down because he hasnt learned how to carry himself. I wouldnt call it a back problem, I'd call it a "downhill leaner" problem. :)
For you, what I would do is when you are riding him, keep him slow. The faster he goes the harder it will be for him to pick himself up. At the walk, give him the feel in your body that you want him to slow even more, and take gently on the reins,(perhaps even a little it of an upwards pressure, rater than back towards your hip) then a slight tug, then an immediate, urgent release. Like, throw your hands forward like he'll do something really really bad if you dont do it fast enough. ;)
What that does, is gives him the aid to steady with the gentle take on the reins. Most likely, that'll be ignored, and the quick tug and then release sets him back and since you arent still pulling, he's got nothing to lean on. The quick tug in the middle is just in a moment, it's not a jerk. You shouldnt feel like you release the reins after the gentle take on the reins, its should all flow together.
Do this every time he feels like he's falling down on his head. If he reacts positively, ten immediately back it up with some leg so send him forwad after he's steadied himself.
This will teach him to carry himself without you having to feel like you have to hold him up. If you can get the owner out one day and tell her the stuff you're learning maybe she can help. :)
Just keep in mind a horse like that will sucker you into pulling on him, but try your very hardest not to hold him up. You can teach him to do it himself.