And also, socks might be binding you in that area too, or jods/breeches/jeans could be, and since so many things are sewn overseas now? Those fabrics are not being cut right, and the cloth pulls funny, and can cause pain.
Also measuring your stirrups, with tape measure might show a difference between them that is causing this.
You might also be placing more weight on one leg than the other, riding crooked, or your horse could be slightly off in way it is going, and you are relating to that when you ride.
Have you gone to chiro for yourself or massage therapist as it could be you are crooked and need straightening.