This horse is just plain spoiled and you are in far over your head.
This is just plain NOT TRUE! There is nothing a horse love better than being abandoned in a field for four years unless he is being abandoned for 10 years.
He over-reacts because there is a 'pay-off' for it. He gets away and get the green grass. This is as much of a pay-off as one can get. He break halters and X-ties and gets the same pay-off. He is calling all of the shots and you are following him around trying to play 'catch-up'.
First of all, stop putting him in X-ties. Buy a good heavy rope halter, learn how to fasten it correctly (put the end around the loop BELOW where it goes through that loop -- not above it) and break this horse to tie solidly. Tie him to something immovable that is higher than his withers. Make sure you have someone experienced helping you and always have a sharp pocket knife with you. This horse needs some manners and you need to learn how horses think and why they do what they do.
Right now, you are not in charge of anything -- he is. It would take a book, not a post here to teach you how to properly get in control, but teaching this horse to respect a halter would be the place to start.