Vet came out today and did the x-rays. Luckily, it is not affecting the joint nor any tendons/ligaments. Though it is close to one. If it gets significantly larger it may be an issue. X-ray showed a line going through the middle of it. Since he had this nearly all his life the vet suspected that as he continued to grow another deposit was formed on top of it. She described it differently but that was the main point.
He passed to be ridden. The only fear is that a good knock could break it where the line is. I am going to be using leg protection. As for jumping, she left that up to me. Though I decided it is best if he doesn't. If he misses a jump and knocks the pole it could lead to injury. Not worth the risk.