Spend time just hanging out in the pasture or paddock. Sit in a lawn chair and read a magazine and don't look at him. They are of such a curious nature, eventually he'll come closer. Lay an old jacket on the ground close by to see if he'll come and explore it. Continue to ignore him and eventually, he'll probably come nosing around. Also, when you can catch him, don't just slap the saddle on and ride, just groom him, even massage him, give him a carrot and then put him back in the paddock. Do this a few times and he'll probably look forward to your visit.