If the tack is new and cheap/stiff, then there's not a lot you can do about it. You can try soaking it in
pure Neatsfoot oil for a couple of hours (warm the oil), but I would NOT soak it overnight. Work the leather in either direction (rolling/bending) as it softens to help work the oil in deeper.
If it's used tack and good quality, then I would NOT use oil. Instead, use something like
Passier Lederbalsam. Warm it and work it in using your hands. Roll/bend the leather as it softens, applying more of the goo as you need it. Let it sit over night, then work it again with more if needed. Let it set one more night, then wipe off any excess.
Oil can harm leather in the long run. If it's cheap and stiff, then it really doesn't matter. However, if the leather is quality just dry, then you want to try and be as "nice" to it as possible, so it will last. I have a pair of 35 yr old draw reins that are still in great shape and a 15 yr old bridle that has not needed any repairs! (and it's been used HARD for all of those 15 years) If you take care of your leather equipment (and it was good quality to start with), it will last a lifetime.
Before either of the above, make sure you clean it thoroughly!