You CAN ride in whatever you want! I went through a phase for a couple of years where I almost always rode in these super thin cotton pants that I had, and I loved them. They weren't designed to be horse breeches and had a loose leg and an inseam, but didn't bother me. I loved them, and was not self-conscious going out in public in them. A couple of years after I bought them they wore giant holes in the crotch within weeks of each other and I haven't been able to find anything similar since :( A quick google search comes up with these, and I'm sure there are plenty of other options: Amazon
.com: Ladies Sigma Cotton Knee Patch Riding..."> Amazon.com: Ladies Sigma Cotton Knee Patch Riding...
There's a variety of purpose-made equestrian breeches in various natural and synthetic materials. It's all trial and error to figure out what you like!
Now, if you're talking showing then things are a bit different. You'll likely be able to find a material that you like regardless, but the discipline in which you are showing will pretty much dictate the style of breeches that you wear.