Yes, let it absorb as much as it can, then stop oiling. Let it sit overnight, then buff off any remaining oil that hasn't absorbed.
If it feels a little tacky after buffing, set it outside in the sun for a couple of hours. That will dry up any residual oil.
Can you tell this is something I've done more than once?