And I thought it was just me! I ride western and have the same problem.
(Do you ride english or western?)
I never noticed it when I just had a dark brown saddle. But now I have a couple of lighter saddles (medium oil) and the sweat marks show. I'm not sure if it's just sweat or if it is also friction from my jean seam.
I'm not sure if anything can be done about it. Anyone else have any ideas?