I would find a girl to come watch your kiddo while you ride. Maybe trade rides or lessons on one of the more broke horses? Maybe there is a neighbor kid (10-14) who would sit with him in a safe area. I know there are thousands of girls that would LOVE to just get to be around a horse, maybe even for free? I would have.
I live out in the boonies so that wasn't an option for me. In the warmer months I would call DH and tell him I was riding, put the kid down for a nap or with toys in a playpen on the porch and ride around in the yard. DH also comes home for lunch so I'd get that ready, tack up a horse and be waiting for him to come eat.
It's sweet that you are letting DH rest... But you need to keep up with your riding for your sanity. They also need time alone to bond. Even if it's just once a week for an hour and a half, that's not too much to ask! I was a SAHWM, so I was working, running the farm, the house, taking care of the kid... He went to the office from 9 to 5, but had to work most weekends. So after a few melt downs I started putting my foot down and taking some time for me. It's best for EVERYONE!
Hang in there!