That sounds like he is still very young, if he is that small. Are there any breeding farms nearby? Perhaps they have a mare who can feed your foal. I've always thought you couldn't feed foals regular milk, I was told it will make them very ill. But if that's a vets opinion, then I'm not sure.
The plants: Grass is the best food for a horse. Some plants are okay, but I strongly suggest fencing him in somewhere with decent grass.
My best advice would be to get the vet out as soon as possible. He/she will give a thorough examination. Chances are they will know if the foal is missing from another farm. If not, they will give you all the instructions you need on feeding and care. Foals (well, horses in general) are expensive, be prepared!
How much experience do you have with horses? Foals learn bad habits fast and as they grow, become dangerous. In a herd of horses, they will rear, kick and bite in play. Generally they do this to humans as well, until they are taught otherwise. If you aren't experienced, I suggest getting some help from a trainer or an experienced friend.