I had this same problem when i bought my horse four years ago. To overcome the problem i sat in the middle of his paddock with a book and ignored him. In time he came over to see what i was doing. I carried on ignoring him until he touched me or nuzzled me 1st. Then i would stroke him once and go back to ignoring him. Over time if your horse thinks you arent interested or if he doesnt see you as a threat by keep trying to get near him he will learn to trust you and eventually come to you. My horse now comes to a call or a whistle every time!!