Just sit the other way around in the saddle, problem solved.
If the horse is scared of stuff then probably a way to get it sorted out is to build its confidence in you. This has nothing to do with “just spending time with it”, and “bonding” with it by giving it treats, pats, or grooming it. What you can do is get something like the Parelli lateral lunging gear and work on that, then transfer the same kind of thing to exercises you can do in the saddle. Now I know Parelli is a dirty word to may around here, but the older stuff is really worth a look. I learned that stuff, but never learned any for these 7 games, just the principals and the techniques to get a horse moving, then made up my own games from that and it works well. Get the horse going around you in a circle well (just do about three circles on each side in any one go, don’t run it round and round and round), then build from there. Look for challenges for the horse to do. So once it can walk around you, get it to walk around you and and between you and the rails of a yard, and after that, through a gate. Or get it to jump over a gully, or a log, get it to walk into a shed (if it is safe to do so) and out again (getting a horse to willingly walk into an enclosed space and be happy about it can be a big step to sorting out fear issues). The thing to remember is 1: all of this may seem disconnected from the instant in which the horse starts to play up, just remember that a good foundation in having that horse trust you and follow your commands will lead to you being able to override the fear response that the horse may give when it would ordinarily act up and 2: that you should start off with the easiest thing to do, just walk around you in a yard or the paddock, and only increase the difficulty/challenge (and so “scariness”) in the tiniest increments that you know the horse can achieve. If you give the horse clear, consistent and firm instruction, it (and you too) will come to see that the horse can do everything you ask it to do (because you set it up to succeed with each challenge) it will come to trust you and it will look to you for security and not other horses around it, and it should put a dent in the jogging backwards issue.
If horse isn’t scared of anything, then Id be looking to see if its confused about something. For example, I have found that a reasonable number of the horses I have trained will walk about backwards on their first ride. So the very first time I get on them I will ask them to walk out but they start going backwards because they are a little confused about what I was asking. Haven’t had it happen in a long while so, now that I think of it, it might be that I'm setting them up better through he ground work these days, but anyway, when they did it I got my signals clear and they walked out forwards once they got what I was asking for. How that might apply to your situation? Keep track of when exactly the horse does it and see if there is anything you are doing that is causing it. If you notice that the horse does it after the same signal from you each time, then you'll probably narrow down the catalyst for it, and be able to figure out a solution. Possibly clearer release of pressure to help sort it out would be my first guess.
If not, if the horse is doing it because it figures doing it will get it out of doing what you want. In that case, start working on respect training from the ground up. The good thing is that all the stuff about the lunging and giving the horse little challenges that I mentioned above will also build respect if you do it right. Also, when you are riding the horse, and it dose this stuff, I wouldn’t try to kick it out forwards, nor would I try to back it up till it gets sick of it. Doing the former seems to be leading to the horse rearing, the latter is kind of like my friend from primary school. He started smoking at 8 years old, if his father caught him he would make him smoke a whole pack of cigarettes in one go. My friend loved it, far from stopping him smoking, it encouraged it.
What I would do is: on the ground get your horse yielding all quarters well, and combine them through side-passing too. Once you have done that, get on its back and get it yielding all quarters well from the saddle, and side passing. Once its going ok, if it starts this jogging backwards stuff, turn it around its back feet by giving a direct rein signal to throw its front end over the appropriate back foot (this will take some practice to get the timing right as you will have to coordinate your signals with where its feet are at to get it to work) and once it comes about, get it moving in a forwards direction before it can seize up and try to rear up on you; after that you can turn it around and set it in the direction you want to go, and repeat the process if necessary. And don't give your horse cigarettes, terrible habit :).