If it means that much to you then just go do it - don't let the opinions of others on this forum sway you from it. His conformation seems better than a lot of other horse's out there - and just because he may have some flaws on his legs does not mean he is going to make a bad riding partner.
However, if you want to acquire more, do you plan on breeding?
BUt I have to ask: What is your experience level? "Rescues" horses often need to be re-educated from an experienced horse person. And just guessing from the picture, that horse may more than likely have a "hard-mouth" which can make it difficult to train the rest of his body, if that makes any sense.
But anyway, he would sure be a pretty horse to have :)