Lots of geldings are like this, even if they aren't gelded late. Removing the testicles does not remove all of the testosterone, he can still be interested just unable to do anything about it.
If you want to know if he was gelded late, look at his sheath and compare the size to other geldings vs. stallions. Usually if gelded past the age of 2, they will have a larger and fully developed sheath. Geldings will have a small sheath if they were gelded before 1 year of age, or before their testosterone starts kicking in. I have a feeling your gelding is exerting dominant behavior around a mare in heat, and that he's probably a dominant gelding anyway. Either way, don't let him get away with that stuff while he's haltered and working. Well behaved stallions can trail ride with mares in heat, there is no excuse for how he is acting.