I figure why go to a less harsh bit when he barely listens with a harsher bit?
Because he will eventually get desensitized to your harsh bit, and then you will have to get one that's harsher, and harsher, and harsher. If you had to yank down on him to get his attention two-handed in a curb bit, you are pretty far gone. I would go straight into a rope halter or a snaffle and work on sensitizing him to the bit, not desensitizing.
If a horse is ramped up and rearing when you are pulling back on him, pulling back even more is worse. At this point I would suggest getting off and lunging him in front of the gate, then let him rest at the far end of the area.
I would first invest in some natural horsemanship ground control. Hip disengagements and casual circles in particular. These are tools you can use when you ride to get your horse's attention back on you. During a hip disengagement, it is very difficult for a horse to rear or bolt off, and it also forces them to work extremely hard in a small space.
I would never, ever yank on my horse's face, even to "raise the energy" to get their attention. The last thing the horse needs is his energy to be raised. The horse needs to spend his energy, and a lot of it, through soft hands and a given direction. If I want to get my horse's focus back, I do hip disengagements until the cows come home, and then I do some more. I want those back feet to be spinning and crossing as fast as they can, but also to be supple with my inside rein and give-and-take appropriately. The good thing about hip disengagements is that you can do them one-rein in a halter, and easier in a snaffle than a curb.