Don't give up, just takes time, patience & a good bit of determination. I can totally relate to the sister thing, my younger sister has always been a perfect size 0, even after having 2 children, it's disgusting. She was the golden child when we were younger, but because of that I gained a lot of grit and determination and a bit of an "I'm going to show you and make you eat crow" attitude. Because of that I am the one who now owns & runs the family farm and has a successful breeding & training operation.
Don't give up! When you go to ride next in your head choose your attitude, go in with no expectations & look for small successes, even if it's getting a few nice strides at the trot or he gives in to the smallest amount of pressure when asked, any simple little thing. Look for those little successes and they will eventually become one big success :)
Life is like a camera. Focus on what's important, Capture the good times, Develop from the negatives and if things don't work out, Take another shot.