Sounds more like a hoof or leg issue than a back issue to me...
I have had a horse that had severe and I mean SEVERE back pain [nearly fell to the ground when you touched his back because it was that bad!] and he was never, not even at his worst, struggling to hold himself up for the farrier. However the TB filly I own now, when she had a minor bout of laminitis, literally collapsed when I picked up one of her front feet. She doesn't have a very high tolerance for pain and that bout of lami combined with her girthiness [she had a saddle on and girthed up at the time] resulted in total collapse.
My current gelding gets sore in his loins and can struggle a little to hold up his hind feet at times but never pulls away or shakes or collapses... he just leans on me. I won't let any professional farrier anywhere near him because of it, too many of them give a horse what-for for leaning and Monty can't not. But when he had a hoof issue [bad cracking] he nearly fell over when I picked up the opposite foot. Currently he has a knee issue, arthritis I think, and is great to pick out but difficult to trim because he struggles to keep his forelegs bent for long enough for a full trim.
I would eliminate all variables. Post hoof pictures for us for critique, have a saddle fitter look at the fit of the saddle, have a bodyworker of some description go over the whole horse for sore spots, etc etc. I like equine chiropractors but there are many kinds of bodywork. Acupuncture works really nicely too. I'm not a fan of Bowen Therapy but a lot of people swear by it.
Oh and one more thing. Just because MY experience indicates that back pain doesn't cause the issue you describe, doesn't mean that it can't. All horses have different levels of pain tolerance.