So I then hopped off, hand walked him up and down the hill about 5 times so he can see there is nothing to spook at, and he was fine. So I got back on him, walked him back into the arena, to get him safely under control again and walked out, we got maybe to the top of the first slop when he started backing up. Usually if he backs he grabs the bit and takes control. So I was quickly able to stop him from backing by putting him in a circle until he let go of the bit and moved forward. I took him out of the circle and walked him forward to the top of the slope. Again once we were there he started backing up. So I decided to try something different and had him walk down the first slope backwards. We got down to the bottom of the first slope until he once again realized we were going back home and started backing up once again.
After finding out backing didn't do any good I walked him back up to the arena and got him quite and controlled again then started to walk him out slowly and each time he would threaten to back up I circled him back into the arena and repeated. Each time he was going to back up I repeated and each time he behaved I let him go forward, mind you this was for maybe only 5 steps before he would start backing again.
I tried this for just over an hour and I hate to say it gave up, partly do to heat exhaustion and hopped off for good and hand walked him back to his stall, which he acted just fine going down the hill.
Mind you this was after a half our of light he has arthritis in his knees and navicular in his leg, and is sound enough for trotting and light canter work.
Has anyone dealt with a horse with this kind of issue? And any idea what else I can do. He also does this out on trail, so I don't go out anymore but I can't just not go out into the arenas like trails, so I have to somehow break him of this habit which also feels a bit like a power struggle and when he does this, he's a completely different and unpredictable horse.