I've ridden plenty in jeans with buttons on the back pockets and I've never seen any scratches. Suede (or whatever that stuff is) seats, slick seats, seats that were just leather over wood, seats that were leather over padding, english saddles...no issues.
That being said: if you're riding in a saddle that you would be broken up over, if it were to get scratched, I would find a different pair of jeans to ride in. Better safe that sorry!
Fabio - 13 year old Arabian/Lipizzaner gelding
Hazel - 14 year old Angora goat
Atticus - 4 year old LaMancha/Alpine cross goat
Rest peacefully, Lacey.