Like you said, it sounds like he doesn't have much respect for you.
I would be very consistent every time your around him that he gives you your space.
As far as the lunging goes, I worked with a horse who did this to me for a while. The horse belonged to someone else and she always did a terrible job of lunging him and letting him get away with bad behavior on the lunge line. As your horse does, he dives in at certain parts of the circle and tries to run me over. What I did--and sounds like you're doing--was smack him hard on the shoulder with the lunge whip as he dives to you. If he doesn't listen, get him on the head. He may then act head shy but after awhile he'll learn to respect you.
Another thing to work on is shoulder and hindquarter control. First work on his hindquarters. You always want him to disengage his hindquarters when you use your body language to drive them away. After you have that mastered, you need to get him to move his shoulders all around. This will help with the diving into the lunge circle. I prefer to do this on a shorter lunge line so the horse is about 5 feet away from you. Be CONFIDENT. With you at his shoulder about 5 feet away from him, send him out to circle you but don't let him go out too far. Here is when you want him to arc around you and move his shoulders away from you. Always advance towards his shoulders. If he isn't listening, shake the line or tap him on his shoulders with the whip. This is easier if you have a dressage whip because sometimes the long lunge whip gets awkward. The point of this exercise is from him to move his shoulders and for you to keep your personal bubble.
You can always mix this up a bit by suddenly asking him to disengage his hindquarters and then send him out again moving his shoulders.
Opportunity is missed by most people because it is dressed in overalls and looks like work. ~Thomas Edison