First & foremost, if you aren't experienced in training horses, especially if you have no idea how to go about it, I cannot stress enough to find a good trainer to at least work with you & the horse. It is very easy to mess up a horse if you don't know what you're doing, & spoil his chances of a good life due to 'bad behaviour'. That said...
I've got him comfortable with a halter on and being led, but other than that I'm kind of stumped. He freaks out when taken from the mare.
Start at the beginning. Don't tie him up until he's really good at yielding to lead pressure in any direction & don't tie him solid until he's good & confident at standing 'tied' with a long lead looped around a rail, with a 'Tie Ring' or some such.
If he isn't used to dealing with humans, then he will naturally be much more reticent to leave his 'herd' & comfortable environment with you. So I'd stick to training him in his pasture with his mate for a start, until you establish a good, trusting relationship with him. Once you get to the point that you're ready to take him into different environments, you could start out taking the mare too, &/or you could start taking him out for VERY short stints - eg start out going out the gate 10 metres, or only a minute & then back, repeating this to gain his confidence with being out with you, & gradually increase the 'journeys' as he becomes confident & learns that it's OK to be out with you, regardless of the other horses carrying on.
I also like to use positive reinforcement & in this situation it would help him see going out with & paying attention to you is a Good Thing for him. If you don't know much about +R training, read up on the principles of 'clicker training'.
I figured I should train him to lunge first, but then I realized I need him to just be comfortable being lead wherever I take him.
Yes, you need to start at the beginning, which is not 'lunging'. Depends what you want ou of 'lunging' as to why you might do it at all too. Think about the basics - the foundations he needs; To learn trust in you & be desensitised/accustomed to what you want, whether it's what you do, where you go, what you use, etc; to understand what you're wanting of him, of which IME yielding to pressure in a variety of ways is the main 'foundation'.
So you've got him comfortable with the halter & being led. Good start. Is he comfortable being touched all over, including being touched with ropes, a stick/whip, etc? I'd work on that first. Then when he's confident with all that, start applying pressure in different ways - with your fingertips, with a stick/whip, swinging rope, etc, to get him to move in various ways - move his forehand away, hind quarters, go forwards, backwards, etc, etc. The point is to *teach* him to *yield*, that is, move softly & willingly with understanding, rather than just try to make him do it, which can lead to him reacting in fear without thought.
Once he's yielding to pressure in all ways, leading really well up close, then you can 'test' it by asking him at gradually bigger distances away from you, until it becomes 'lunging', which is, to me, just for teaching/confirming these basics at a distance.