Update: The previous owner came out today and seemed genuinely shocked at the description we gave him of how Sarge behaved with us when trying to saddle him.
He gave us a complete run-down of proper saddling techniques, bridles, fit, etc. As he went to mount the horse, he commented on how Sarge's ears were turned back and since he felt that he was nervous for some reason, he just sat there in the saddle with him for a few minutes. After taking off with him, he rode beautifully, no problems whatsoever.
Then it was our turn (gulp!) but, just the same, Sarge was perfectly fine and a smooth rider. Exactly how he was for us when we first met him at the previous owner's farm. Strange, no real explanation other than perhaps our techniques were not completely correct and that he was just uncomfortable with us and his new surroundings at first.
My husband saddled up Sarge again several hours after the previous owner had left (just to make sure his calm disposition was not just due to the other owner's presence) and again, he did terrifically. My husband even went on to say that Sarge is a smoother ride than our Tennessee Walker!
I can't tell you how relieved I am. Also, the previous owner has agreed to come to our place 1 or 2x per week for training lessons--and EXTREMELY reasonably priced at only about 30 bucks. So we are completely thrilled.