First & foremost, especially as he was apparently born like this & it's apparently not a 'default' genetic setting, given his laid back siblings, and he does indeed sound overly 'nervy', I'd look closely into nutrition, ensure he's getting high magnesium, high salt, low potassium, low calcium(in relation to Mg).
I'd also get a chiro vet to check him out - pain/nerve problems can cause an 'overly nervy' horse. Perhaps get his eyesight checked too. Obviously he may not be up to that yet, but I'd make it a priority as soon as he's there.
Around four-five months old, I tried halter/lead breaking him. Everyday for an hour I'd work with him just to take one step forward and most times he'd refuse and throw himself bodily on the ground.
Of course, depends how exactly you were going about it, & how you went about 'hands on from the time he could stand' as to whether that is likely to have helped or hindered. But on face value, the above sounds to me like you were just asking for way too much. If you were not getting anywhere, causing(albeit inadvertently) the horse to panic so much that he threw himself to the ground, this is likely to have caused him MORE fear about the goings on. It's always best to work in very short(even a minute to begin with) sessions with green horses, and keep it non-confrontational, so you can build gradually on successes. So if the horse isn't even confident being near you, I wouldn't have attempted to halter/lead him yet.
At any given time when desensitising, aim to 'push' a horse enough only to the level of discomfort, not into the realm of real fear/reactivity. Back off *before* he gets reactive. Rinse & repeat until he's confident with that level & his 'comfort zone' is extended. Then repeat process pushing him a little further than his new comfort zone.
He was still touchable, but wasn't a very 'touch me' horse and would leave after two or so strokes.
So for eg. if he can tolerate 2-3 strokes before it gets too much for him, I'd start out approaching him & giving him 1-2 strokes and leave him before he feels the need to leave. I'd also personally add some positive reinforcement too. As a good scratch would obviously be punishment, not reward for this horse, I'd give him a small food treat when he allowed himself to be touched too.
His dam and half-sister live side by side with him in two separate paddocks. He doesn't exhibit normal herd behavior. Whenever the girls nap by the fence he's alone by himself on the other end of the paddock. In Nov (2014) he decided to bolt through the fence that separates them and ended up with a good 2-4inch flap on his face as well as knocking down three panels of fencing. The odd part? The girls actually ducked under the electric braid (that was untouched/broken) to get away from him.
I'd suggest keeping him in a herd, not solitary. Of course he doesn't 'display normal herd behaviour' if he's not living 'normally'. And the more stress in his living environment - even if 'low grade' from living alone - will effect his behaviour, as well as health. Will also increase his need for Mg. When he ran through the fence, something probably frightened him acutely(& if the camel's already loaded, it may have only taken another straw), and he unthinkingly bolted for the safety of other horses. I don't think it's that odd that your girls escaped from him, if he's so edgy & needy, he probably made them uncomfortable.
When you finally manage to get him clipped and standing still he will turn his head away from you and constantly looks away at his escape path. I feel like he doesn't want to communicate and isn't 'in the moment'. When you work with a horse you expect them to be further than Step 1
Perhaps because you're expecting a certain 'progress' & not getting it(so he's
absolutely 'in the moment' but YOU
aren't), sounds like he's being confronted with too much so of course he doesn't want to 'communicate', but his mind is on nothing but escape/self preservation. Instead of thinking about what he 'should' be up to, take the horse as you find him, where he IS.
Now if you had read this and I didn't say, "He was born and raised on my farm," you'd think he'd been abused.
Not at all actually, but yes, many assume abuse when horses are reactive, headshy, scared of whips, etc. But he does sound abnormally edgy, regardless of handling. As said, I'd start out assessing/amending nutrition and keeping him in a more natural, low stress herd environment. And then greatly reduce training expectations & do things in less confrontational ways. Can you find a good trainer or horse person who is understanding of natural equine behaviour & can help you?