Old horse has bad attitude...need help before someone gets hurt!!!

I have a 18-20 year old haflinger cross gelding horse. He is a sweet horse and loves to be brushed, but he has an attitude sometimes.

First off he has NOT had any attention or been ridden in 6 years! The people I got him from just let him sit in a pasture for 6 years. He was given to me by those people, because they didn't want him.
I am not an experienced horse person. Please don't tell me get a trainer! I can't afford one. And please don't tell me ride him more! I have ridden horses before, but this is my first horse.

He is old and set in his ways, but has sooooo much potential in him.

He is taller than me and only respects people taller than him.
He is a very smart horse. And even if you show him dominance, he is smart enough to know that you are smaller than him.
You can ride him while a taller person leads him, but that's not what I want to do forever. He gets into the little attitude moments, that are very, very, very hard to stop. If he wants to do something then he will do whatever possible to do it!

Please give me some advice on what to do. And what to do to break his attitude.

Sorry for the long post

Why accept a horse that you cannot control? You said yourself that you are not experienced with horses and here you are taking on a horse that pushes you around because you're not tall enough?

The only thing you CAN do at this point is get a trainer, unless you would rather put yourself in danger. If you can't afford a trainer, don't take on a horse that needs one. I would honestly give him back to whoever gave him to you and spend the money you were going to use taking care of him on lessons, then lease a horse, and THEN buy one. Eventually this horse will break down your confidence and you could get turned off of horses forever. I've seen it more than once and it's almost always because the owner/rider took on way more than they could handle.

I know it's not what you want to hear, but when you're working with animals that weigh over a 1,000 lbs, safety should come before everything else.

I think you've bitten off more than you can chew for a first horse. I'd suggest finding him a home where they can handle his issues and get a more experienced and docile horse that can learn with. You're not going to get that with this horse. I know it's a awful thing to hear, and when people told me that about my first horse I got angry and defensive, but in the end it's the right thing to do.