I am 61 and have been riding bareback since I was ten. I still ride all of my horses without a saddle, unless we're in a parade
While I am retired from "Calgary Stampede" riding, when I was young I thought nothing of taking my Arab/Saddlebred down power lines so steep that he literally sat on his butt and walked his behind down, then would dig with his knees the last 20 feet up the other side.
I prayed both ways that I wouldn't slide off one end or the other
He was around 15.1, tough as nails and I only weighed 120, so he barely knew I was on his back.
I raised that Arab/Saddlebred from birth and we did parades (I used a saddle for that) and trail rides until I had to put him down with cancer when he was 29. He was no where near looking sway-backed.
So if the balance is truly there, the horse won't be hurt --- just be sure to not "ride the horse's mouth" when riding bareback.