I suppose you can make it pretty much whatever you want it to be.
Around here there are several groups that organize trail rides, and a bunch of people meet up at those trails. I would call those riding clubs.
Sometimes they might also have non-mounted meetings, put on an occasional competition, hold seminars, have speakers, etc.
Really it's just a group of horse people who like to get together for whatever activities the club has defined.
Unless there's some more precise meaning than that - that's all I'm familiar with.