Sometimes Ice's hooves get a little packed in as well and it takes me FOREVER to get stuff out (this doesn't make him happy--he's used to professional grooms who can do it in like two seconds, lol) but as long as you know the relative anatomy of the hoof, I.e. Where the frog is, you should be able to be a little rough on some spots and maybe make a hole for yourself. I usually start in the little pockets on each side of the frog near the heel, where dirt usually is the deepest and try to dig stuff out from there and work out to the toe.
I don't use a damp cloth, but I do use these
Oster® Grooming Towels - Finishing & Grooming
Since Ice doesn't like me brushing his face, and he also has some tear problems, theyre perfect. They do smell a little bit, but they make him shiny and smooth. You can use them just about anywhere on a horse too!