I think RiccilOve got it right. You haven't failed at all. This horse may have had negative experiences with people for a very long time, such that he has just "shut down" in his ability to relate and be interested in people. I've seen this many times -- usually in horses that are "well trained", meaning that they are pretty push-button to ride. Used to see this "gone" kind of personality on the hunter circuit a lot...one of the reasons I quit that discipline, as I knew what it took to create that flat, dead aspect in the horse. Such horses typically do their job, but always with the sense about them that they're doing it because they have to, not because they want to. They tolerate the presence of people for the same reason. They have been given no reason to feel joy about us.
Can this psychological damage be undone? In my experience, sometimes yes, but it generally takes a very long time and a total change in what and how the horse is asked to exist and function. Sometimes, it just is what it is and it stays that way.
We had a horse like this for a while -- bought him for my newbie rider husband because he was push button and safe. Had the personality of a wet rag. Even my other horse found this unlikeable (he hated that horse and wanted him nowhere near him). Who knows what that horse was like initially, before people turned him off like that? He was a good horse to ride, but we also like to have a feeling of relationship and enjoyment coming to and from our horses, so we didn't keep him. Sold him to a trainer who was coming back after very serious injury, and she just loved him. She also used him in her school string, and he was a favorite with the kids because he was so easy to ride. So, he was a perfect fit for her needs -- not so much for ours.
In your case, you can either give this horse some time to come around and accept that he may or may not, or just find him the right home and find yourself the right horse. If you do want to try keeping him, perhaps you can tell us what you know about his previous life (use, training, housing situation, etc.) and how things are now. Maybe I could make some suggestions as to what might help turn him around, though like I said, he may never change.
Bottom line, you are clearly a caring and loving horse owner, so this is no reflection on you, my dear.