I have neighbors directly across the street, and directly north and south of my 5 acres. I suggest the letter and filling your freezers. That is just plain rude to use your property to feed/house their cattle. I'm sure that you can be polite and tell them that the new owner has moved in and the old deal is now moot.
I have to agree with the others. The neighbors who had the cattle had to have had plenty of warning that the property was being sold. New owners have no reason to want someone else's cattle on their property. They should have been making plans to move their cattle well before you bought he property!
It sounds very much like they are taking advantage of you. I would tell him to come get his cattle and rent a bobcat or hire someone else to do the work.
It has already been 3 weeks. Your neighbor has NO reason to do what you are asking. He is getting something for free by not doing what you ask. Why would he change that? He has shown disrespect to the previous owner by letting his cattle get things into such a state that it needed repair.
Have you also thought about the potential for liability if his cattle cause damage...or if something happens to his cattle while they are on your land?
IMO...set a line not to be crossed and set it now. Good fences make for good neighbors.