Do you have better pictures? The upper hoof wall is nothing to worry about, it's actually good. It's new hoof growth that is growing down. There's a difference because of the laminae attaching the hoof wall to the internal structures of the foot. Flaring and such can pull the laminae away from the hoof wall, like pulling velcro apart. The new hoof growth is "new velcro." It's tighter and has not yet pulled away from the hoof wall which it should not do in the first place. The new hoof growth will grow down in the next few months and replace the old hoof growth as it's trimmed away.
As to why he is lame, I wouldn't know. X-rays would be your best bet on finding the culprit.
There on the tips of fair fresh flowers feedeth he; How joyous his neigh,
there in the midst of sacred pollen hidden, all hidden he; how joyous his neigh