In my experience, (I worked at a gravel pit) stone dust or crusher dust can turn into a slimy mess when wet, depending on the rock used. Then when it drys can set up much like cement. I definitely would not use it as arena footing. People would buy it to set their patio pavers if that gives you an indication.
Washed sand that is free of those smaller particles will stay freed up when wet and will not become hard if you maintain your base and are careful working it up as not to rip the base up and mix it into your washed sand.
I DON'T LEAD 'EM AND FEED 'EM, I RIDE 'EM AND SLIDE 'EM.