If I were in your shoes, I'd try to talk to the trainer, and see if he/she would agree to just once a week riding. Then the rest of the time you can work on ground things or lunging, etc. I guess it would depends on how stable he is. Is he a good well rounded horse? (I know you said good boy.) Or is the trainer really working on things with him? If the trainer is just putting saddle time on him, then I think it would be ok to just do a weekly riding. Or do you have a close friend who you trust in riding? They could do a day a week, too.
~*~Never approach a bull from the front, a horse from the rear or a fool from any direction. - Cowboy saying~*~