Is your horse slow? Are you pumping to subconsciously drive your horse forward? Regardless of why you do it though, you are driving him forward, and your toe periodically turns out and jabs our horse in the side with your heel. I would say really work on strengthening your leg in a correct position: even contact through your thigh, knee, and calf making sure your knee isn't pinching (which will cause your lower leg to swing back and forth or your toes you swing out and in) and making sure that you're always riding off the inside of your calf and the weight on the inside of your stirrup. Riding stirrupless will definitely help out a lot! That would be my biggest recommendation. Riding in a correct 2 point (or half seat) position will also be good. One step further is an exercise Geoff Teal describes in one of his books: Get up in a half seat and squeeze your thighs and calves on your horse, but not your heel. It's an isometric exercise so you're just applying pressure, your horse shouldn't go any more forward. If your horse jumps forward when you put on pressure you're digging in your heels. If you do it correctly it'll start to burn after 10 seconds or so. :) And you mentioned that you're already doing it, but continue working on keeping your hands in contact with your horse's mouth so your arms move back and forth (independantly) and your upper body stays still. It's a common fault! Hope some of this helps a bit!