Sounds like the time he spent unhandled has turned him a little 'wild' for want of a better word. The best advice I can give is to go back to basics with him. Depending on your experience level it might be an idea to send him to a trainer to give him a refresher type course.
Ponies especially can be high maintenance at the best of times but after being left for a long time as you said, it can make them even harder to manage. Stay safe and get someone in who can retrain the little fella.