It also could be that at this horses age, and your weight, that there are some major pain issues going on. I am concerned that you feel his being ungraceful and your being large might flip him over.
Might not have anything to do with hackamore, but just popped up at same time.
But if horse is doing what you describe? You have no business taking him out on trail rides period. You can't control him, and he is a danger to others and their horses too.
Teeth, jaw, bars, ears, poll? Vet check by a equine vet would be what I would do. But I really think this horse is just too old to do the cantering and the long rides, especially if you are a larger human.
Horses make me a better person.