TXhorseman nailed it. You're not balanced in the saddle. It's possible you're just riding in a saddle this is either unbalanced on the horse or not balancing you well. Anyway, a good fix to find your balance is alternate sitting, posting, standing straight up and 2-point when trotting. Post at all different intervals. Stay up 2 extra beats, than 5, than 2, etc. It really helps you find your balance point over your feet. So does the standing straight up.
The other good exercise just to tighten things up in general is to ride without stirrups.
You just have to see your distance...you don't have to like it.