I would let the horse decide within reason obviously. It doesn't sound like he is trying to run around as fast as he can when you put him in, and you are paying attention to him showing signs of being tired. Once you feel comfortable, and if you can, take him up and down some hills to help build up his butt. It sounds like you are doing a good job getting him back into shape slowly. Can't wait to see how he turns out.