This happened to me (if I processing your description right) for awhile on my horse. One reason was because my hands were steady without being in frame, but once I got in frame they started vibrating (stupid hands!) later on, the reason was because I was using a little getting into frame, but stopped once I got it. Then when we slowed down I started squeezing again and Lennon took that as a "I'm not going to do it" kind of thing. Or, you're not using enough leg in the first place and when you get into frame you don't have enough energy to stay there.
Remember, always back up your hands with your legs... Actually back up your legs with your hands is how it should be said.