Umm, I would PROBABLY find a different boyfriend. You can tell by his comment and actions that he clearly is not the horse person he says he is, and he is disrespecting you.
There's a difference between "not being a fan of horses but still respecting them" and "strongly disliking horses and being rude about it".
My boyfriend of 2 years isn't a fan of horses, but he respects the fact that if I don't go to the barn to see Image after work I'm going to be very cranky. He hardly comes to my barn, but when he does he plays with Image and is nice to him and gives him treats. If I need help cleaning stalls, no matter how much he hates it he will always come help me when I ask. He refuses to ride a horse and I can respect that; it's just not his cup of tea. Just like playing Dungeons and Dragons isn't my cup of tea, haha.
There's a fine line between not like something, and just plain disrespecting you.