Just my thoughts on the issue...
I actually think that maybe his bad behavior is beginning to spark from always 'working'. You almost have to look at is as if you where in his situation. He's associating you with the fact that every time you come to see him, in the end he's going to have to give you something.
I can't even describe how much progress you can make with a horse without even putting a saddle on them. Just you and them, being together.
My horse Misty has been through a lot since I bought her. We bought her in September of '06 as a 4 year old, not knowing she was pregnant. The next year I was so wrapped up in establishing a relationship between me and her son, Dakota, & she got put on the back burner. Then when it came time for weaning we put her in her own corral next to Dakota, until Feb. When we took her out to my friends stable. I would go see her regularly and just chill with her, and a few times we went to the arena and rode. We only kept her there three months until my aunt and uncle offered to keep her with their horses in my grandparents pasture. She wasn't thrilled about leaving the horses at the stable, but she clicked with my uncles horses. Which was also great, because they are the ones that I do all my riding with during the summer. Then at the end of the summer I had to bring her back home, and that's when it went downhill. She wouldn't let us near her, touch her, wouldn't let us on the sides of her, and would only face up to us. This made even the simplest task of haltering her near impossible. She was very timid and spooky, as if something was going to grab her at any moment. Complete wild horse mentality.
Ever since she's been home, I've been consistantly working with her on our relationship. In my opinion, your relationship is one of the most important things to work on with your horse. Without it, I don't really think they are going to really meet their potential.
Personally, I think that working on the ground with your horse should be just as important as working in the saddle. Now I'm not talking about working with them in terms of lunging them or ground driving... More so just relationship establishing activities. Make them want to be with you, which will make them more willing to do what you ask of them. Not just you sitting on top of them saying, 'do this because I'm your boss and I say so'.
In just a few short months I have gone from a horse that was completely untrusting and scared, to one of the biggest loveable puppy dogs ever. She comes to greet me at the gate, follows me everywhere when I'm in the corral, always wants to snuggle, comes to me for protection when she gets scared, and seeks out a job/task.
What I meen by her seeking out a task is just this. Since haltering her was out of the question at the time, I had to come up with a way for her to see me in a different light, and start believing that I wasn't the bad guy. First, before even entering the corral I would stand there and call her name until she came over to the gate. Now, at time's this took a while, but I didn't go in until she came over. Curiousity usually got the best of her. You have to remember, this is their area. Just like how you wouldn't like people barging in your house un-announced. Invasion of space. So once she came to the gate, I'd let her sniff my hand, I would take one step in the gate, and then give her a cookie. Just my way of saying thanks, and I'm here on good terms.
I would just hang out with her, and in the beginning never reached for her more than my fingertips could extend. It started with just being able to touch her face, and slowly progressed. (She was having major issues with touching her body, so I had to start with her face since that was all she would give me, then work twords her neck, then her body.)
Then, being the cookie loving horse she is, I just did a simple exersize that has got us where we are today. It's alot like leading, without the halter. When she would start getting ahead of me, I would simply turn around in the opposite direction she was heading, twords her in order to keep her next to me, and start walking the other direction. I'd then stop, and she would stop with me. I'd give her a cookie, and while she was happily muching on that I would pet her neck. When she started getting uncomfortable with that, I would simply take away the pressure by either start walking again, or if we were face to face backing up. Then as soon as that pressure was taken away she would go, wait, 'where are you going??', and follow me again. That made it her idea to be with me, not me forcing my presence among her. Now when I'm just standing there with her, you can tell she wants me to do something like what I was doing with her. She walks around me and stands next to me as she did previously for a while, almost as if she's asking me to do it some more with her. Then if I don't after a while, she will walk back in front of me and just look at me like, can we do something?
I took this all very slowly with her, and never tried rushing into things. More times than not, you can't expect a horse to learn something right away. Repetition. Little by little, we saw improvement. And to be quite honest, I never expected we'd be where we are now after that, or atleast anytime soon. She by far exceeded my expectations.
Now obviously it won't be the same in other cases, because every horse is unique and needs to be handled differently. It's important that you spend time with them to figure out why they do what they do, and how you are going to go about fixing it. That's why I think you get to know your horse so well through it. You have to really understand how that horse works, not just the similar horses that do similar things, and work through the problem together to reach a potential goal. For atleast the first 1 1/2- 2 months I didn't even try haltering Misty, because that was not what I needed to do in order to gain her trust again. Of course that was a future goal I had for her, but I needed to gain her trust back before I was going to drag her around like some prisoner in shackles (not really, but figuratively speaking).
So all in all, I guess what I'm trying to say is that your horse may be more willing to do what you ask if you focus on building that relationship back up between you two. It's worth a shot at least. Make it so that when he see's you it's more like a friend coming over, than a figurehead like a boss telling him what he needs to do. Give him a reason to want to do something for you.
I hope this helps some! :) It's just how I've been approaching things, and it has helped me tremendously with everything I do.