Depends what 'weakness' means. Did a vet look at his back and hind end and tell you what was wrong?
It could be conformation and he just needs correct work. Would also mean that he isn't able to compensate for incorrect work like a more muscle bound horse would. If he wasn't been in work, there's nothing to say that what appears as 'weakness' isn't actually an injury, healed or chronic.
If it's just conformation and lack of fitness, then work slow, work correct, and there's no reason you couldn't play around with barrels.