I had this same problem when I bought my horse four years ago. To overcome the problem I sat in the middle of his paddock with a book and ignored him. In time he came over to see what I was doing. I carried on ignoring him until he touched me or nuzzled me 1st. Then I would stroke him once and go back to ignoring him. Over time if your horse thinks you arent interested or if he doesnt see you as a threat by keep trying to get near him he will learn to trust you and eventually come to you. My horse now comes to a call or a whistle every time!!