Is he food-motivated at all?
My Paint is fairly spooky like you describe. He grew up on a private farm with acres of grass and one other mare...for 8 years. He wasn't exposed to much and when I brought him to my boarding barn, he was scared of everything. BUT, he loved treats.
I spent a few days just hanging out with him around the property, bonding and being his alpha mare. When he spooked at something, I'd stay calm, and unzip my pocked (full of treats). When he calmed (even the slightest), I'd give him a treat and say "good boy." Soon enough, when something scared him, he'd look to me for support, comfort, and of course, a treat.
Some people are against using food rewards for horses. Some horses can get pushy or nippy. My horse has always been very polite with treats...and I'd much rather have him dive his nose into my pocket when he's scared than throw up his head and run off bucking!
Of course, all this only has worked for ground work.
My horse too, was afraid of things in the arena such as your flowerbox jump (or our mounting block). I just turned him out in the arena a few times and let him inspect everything on his own. I sat on our mounting block and even though he was terrified of it, he still came up to the sound of my unzipping treat pocket to snag a snack!
Wow, I'm writing a novel. But I also agree with moomoo that your Paint may just need some consistent one-on-one training in order to trust you.