I bet that's exactly what it was. Because of his color he was deemed less than adequate. Which is a shame. As long as you don't mind him not being registered or his foals.
I think he's a fine looking specimen. Some of those draft breeds are super specific about color and I can understand why. And they are not very prevalent in the U.S. Or Canada which makes it doubly cool.
UC Davis does genetic testing: Horse Tests
It may be that he isn't all Ardennais (that's the spelling in my book) and has some other in him which might have been another reason other than color someone decided not to keep him. It's too bad, because he's a fine looking horse.