Horses are emotional beings. They do not always do what we want becasue they have their own mind and their own feelings. Becuase his behavior changes from the indoor to the outdoor tells you that he is very uncomfortable in the outdoor. He is basically scared out there. He's not being good, then bad. He's living in the present. So, when he's in a scary place (for him, at least) he acts on those feelings. When he is in a calm place, then he feels secure (such as you on him bareback indoors) , he acts "good".
Your job as his rider is to not deny his feelings, but work to proving to him that he can follow something other than his immediate feelings; YOU! If you cannot make him obey you out there, then get off and work on making him listen to and obey you out there with you on the ground, and don't make him stay out there a long time, the first time. You work into new and scary things little by little.
That being said, if you are riding him and he turns around, runs away from the scary thing and yo9u let him do this, he will have less and less confidence in you as a leader. If you have to get off, do so but take him back out there and walk him around. Work with him and just hang out out there for a bit. Walk around the outdoor arena looking down at the ground , leading him, acting as if you are hunting for Easter eggs. It is your horse's business to jus follow you, and your being mentally absorbed in something other than his fear will help him to join you in being unafraid. Act as if there is nothing to fear and he will have nothing to fear, eventually.
Spend more time getting to know this horse and doing things that he is succesful before you start asking him to jump, unless this is something you two have already done a lot of before. Work your way up in steps, such that they are successful. Each one will make the next one more likely to be successful, too.
When you have refusals, they tend to make more refusals more likely. But, if your horse was feeling more confident about his surroundings and his rider, he would not likely refuse in the first place.