i'm in ks as was stated before, and a CCL cost a bit, but what's your life worth? Don't be lazy, 50 bucks is cheap. If I recall, GA is one of the better CCW states, or even open carry. Check your local and state laws. I would strongly recommend carrying on your person as well as it is more available.
I carry nearly 24/7, and when I ride I DO have a rifle boot where the old .30/30 resides. Handgun also goes with, but I disagree a bit with eddie. He says single action revolver, I would be more apt to recommend a double action since the trigger can be pulled without needing to dock the hammer. I unfortunately don't have either so I carry a full size auto, which is part of the reason for the rifle.
I do still consider two legged creatures when I ride, although it's almost always on private land, or on a county road. But I still do consider the four legged type. There are cougars around here from time to time, and constant coyotes, as well as a new addition: hogs. Being out on horseback does not mean that I want to let my guard down in any fashion and let a pest go (hogs are new, creeping up in from OK, and we do NOT want any of them at all...).
Like I said, just get off your duff and go get your license and carry on your person. It can be well worth your life to do so, especially if you come across some stray two legged creatures. Something I forgot out around here is METH LABS. I don't want to come up on a meth head and his meth lab with NO gun, they are extremely likely to take a whack at you no matter who ya are or why ya's there.