Seems less likely it's a fracture, since it's a muscular area, He may have just decided that it's more sore now that the muscle has gotten stiff. I was nailed in the hip and thigh by a colt a couple of months ago, and it sure hurt worse after 2 days than it did the day of, and I didn't have a fracture or nerve damage, it just took that long for me to get stove up, so to speak, and movement made it feel better, after I worked the initial stiffness out each morning. It made me gimpy for a better part of two weeks. I had a huge swelling,too, and it turned a rainbow of colors before it went away. A few years ago I was nailed in the thigh and now I have a dent in the muscle, and whew did that one hurt, and yes, it hurt worse a couple of days after than it did the day of.
In blows to the muscle, pasture exercise and handwalking will help it heal better than stall rest, and by now the ice isn't going to do much for him, if it's just a big contusion. Bute probably isn't really helping all that much, either, but it may make YOU feel better. He's likely got all sorts of broken blood vessels and tissue that will just take time to heal.
One thing to consider, since there was broken skin, is a puncture wound, so still should have the vet come out. Again, ice won't hurt but may not really help at this time, and it's a hard place to keep iced down anyhow. Keep it clean, and let him move some. If it were a bone or tendon issue, I'd try to keep him quiet, but it's in a pretty elastic part of his body, so movement now may help it heal faster and better by increasing circulation (blood flow will carry away the cellular debris, bring oxygen and white cells to fight infection) and rebuild the tissue that was destroyed, and movement keeps it elastic. Don't work him hard, by any means, but I know when I got kicked, I ached less when I walked around as usual. Sitting in the truck was harder on me than anything!