I have trained many ponies and not a few horses to drive. Because the potential for bad accidents is higher with a driving horse, it needs to be 100% before it ever gets out in the open, or even driven in an arena.
I think your pony needs to be restarted by a competant driving trainer. You can't train a horse to drive just because you know how to ride. Can you find someone who drives who can help you. If a driver trained your horse, you really need to find someone else.
Since it sounds to me like you are a novice, I would suggest that you take some lessons with a really well trained driving horse.
If you look on the American Driving Society website, you can probably find a driving club near you, where you can get good information and help with your pony.
Driving is a blast, but it can be very dangerous. My neighbour, who is an expeienced horseman and been driving for over 3 years had an terrible accident over Christmas and broke both his legs among other bad injuries.
Last but not least... PLEASE wear a helmet, it could save your life.