It sounds like to me that you have allowed no time for yourself to have any fun at all doing anything. You owe it to yourself and your family to have some time that you enjoy. You will be a much happier person.
I have been in your position before. I have 3 kids, work and horses. I put riding on the back burner, too. It seems like when you have so much to do, that adding riding in to the picture just made everything so much more difficult to juggle. I have learned though, just because I make a few days a week for riding, doesn't mean everything else falls apart. Just the opposite...it may be the very thing that saves your sanity....it definitely does mine. The toughest part is in the beginning and setting aside that time. Mind you, it isn't always pretty. Sometimes I have to drag the kids to the barn with me and ride close by. Your kids are 4 and 8...trust me when I say that they can entertain themselves for an hour. My kids spend time at the barn playing, finger painting,they have a tire swing...they have a slip and slide they can play on, sometimes I even let them play in mud puddles and they also have 3 buckets they can use for picking up rocks for when they decide they can't entertain themselves for an hour (after the first time, rarely will the buckets come in handy again ).
Also, if your children are old enough to be riding, then they are old enough to help. IMO, once a child is old enough to start riding then they are also old enough to help with grooming, feeding, watering, cleaning out stall, etc. Even your 4 yr old can help some. My oldest child has spent way more time helping clean stalls, scrubbing water buckets, feeding and grooming than she has actual riding. It has not hurt her in the least. Just the opposite. She knows that horses are a lot of fun, but a lot of work too. If I had my way, that's the way everyone would learn....but I suppose that is a topic for a different thread.
Stop beating yourself up about not bringing money into the household. Look at it this way, if you had a full time job, how much would child care cost, how much would it cost to have someone tend the animals while you are working and to keep house. Your job is the hardest of all...you can't punch a time clock and leave it...it is 24/7.