I agree. I hauled the last few years without 4x4, and it was always SO nerve wracking. Granted, we have snow 6 months out of the year and the other 6 months are usually swampy mud, so we have a GOOD reason for needing 4x4 where I live. Most of the events I go to are local rodeos with open field parking where it's always guaranteed at least ONE person will get stuck with 2WD!
Now I have 4x4 and it's amazing. We had to yank the trailer out from behind a snowbank in some pretty muddy swampy ground and I threw her into 4-low and it was like nothing, pulled it out like a breeze!
I really think you'd be MUCH happier pitching out the extra money and make your sacrifice in the mileage as opposed to the leaving out the 4x4.