I was going to suggest, as Kayty did, to close your eyes at times. OR, maybe not close them but use the "soft eyes" approach. That is, you kind of glaze over your pin point focus and start looking at the world with a kind of focus that includes your peripheral vision equally. So, you are taking in all the world around you equally, but you are focussed back in your body. So, you see enough of the world to not run into a wall, but you keep your focus internal; to you and your horse and the space you occupy at any one moment.
Doing this will help you to FEEL how the horse moves through space. Like a skier, you can be moving forward straight, riding an edge, or you can be drifting sideways while going forward. Those can be felt easily . Try the soft eyes, or closed eyes, and see if you feel any drift.