You need to work on control. Obviously, even though you're asking for speed, your horse isn't listening. Go back to the basics. Do some groundwork, then do basics in the saddle. Get your horse light and responsive and where he'll move off your leg easily. THEN you can ask for speed.
However, I must say that if your horse doesn't like barrels, he'll never be great at them. I don't like the idea of forcing him to do them. But if he likes them and is just not great... Help him get better. He may never be a 1D horse, but if he enjoys it and you do, just have fun and do the best you can.
I got summer hating on me cause I'm hotter than the sun. Win-ter hating on me cause I'm colder than ya'll. I'm being hated by the seasons so forget ya'll who hating for no reason!