Doing odd jobs or working as an independent contractor do not give mortgage companies or other financial lenders the warm and fuzzies. They want a solid work history, with guaranteed salary amounts. Which means college and a marketable degree.
Stay away from modulars and doublewides if you ever plan to sell or refinance, as they don't generally qualify for 30 year mortgages. Some can, but the hoops you have to jump through to get that mortgage will be unbelievable. Their half-life and worth are also much, much lower than that of a stick built home.
If I'd had the chance to travel before settling in to the daily grind, I'd have jumped at it. You'll be working most of the rest of your adult life, so if you can afford to take the time off and travel, do it.