One month on? Tradionally working mares had their foals follow them shortly after birth when they went back to work. My mare is currently in foal, and I will be taking her foal along from before its weaned on(my mare is an experienced pony horse). My three year old I bought at 10 months and started ponying her then. No tack, at a walk, on an experienced pony horse. If you are thinking of some more challenging trails, definitely wait till 2-3, but for your average terrain, go ahead and start young. I see tons of people in the mountains riding mares with their 4-5 month old foals along side.
I think it depends on the terrain, the weather (heat) and the build / condition of your youngster. I've seen people riding mares with a haltered foal alongside on some quiet, level trails in the early morning hours.
As young as possible. Back when my gelding was born, I started ponying him along with my mare (his dam) when he was only a few weeks old. Now he's seven years old and is an expert at being ponied whenever I need to pony him somewhere. :)