OP .. since we are talking about a training issue, I believe he needs to be desensitized to other horses running, jumping, doing whatever around him. The only way to do that is to put him in that situation.
I would begin by either being there when that is going on, or hauling him to barrel races, etc. and tie him up where he can see and hear that going on ALL.DAY.LONG.
Alot of folks will haul their yearlings and two-year olds to barrel races or rodeos for that reason. They just get used to all the commotion.
Until he's completely bored with all that, don't ride him. Start back with basics, ground work, especially if there is activity going on and progress from there.
Surgery to remove a testicle from the abdomen isn't cheap, so I feel your pain there. Gelding him won't change the behavior, but as others have pointed out, it will be safer for you and for others, especially with the training issues going on.
He's only 4. He's going to get continually more and more frustrated and you're having a hard time handling him now. I would make gelding him priority one.