Maybe its just not something he enjoys and your going to have to just live with it, there's no harm in asking him if he would like to join you or have him around your horses, But you have to respect his decision if its something he's just not into.
I once dated a guy way back when who never liked to go riding till one day I convinced him to come with me, like your bf he seemed to be enjoying himself and never complained, but he never wanted to do it after that. I then heard the grape vine that he didn't want to continue on because he was afraid to fail at it in front of me. Point being there could be a number of issues behind why he doesn't want to do it anymore, your probably best off trying to talk to him about it and see, just don't try n push the subject on him, might do more harm then good.
Maybe there's an activity that he enjoys that you could offer to go and do with him, and spend time doing things he does enjoy, then maybe he will be open to trying out horses again.
I have a non horsey husband, he supports my addiction and love of horses, but its just not his thing, some ppl like it some ppl don't. I don't have really any other advice for you, maybe someone else here does :)