It is all about efficiency and where you keep your supplies, I guess.
I can grain, hay the pastures, lead out 25 horses 2 at a time, clean all 25 stalls, pick the dry lots, throw PM hay in the stalls, bed them all as needed, clean buckets as needed, and prepare the PM grain, all in about 2.5 - 3 hours depending on how tired I am.
I can't imagine that cleaning only 5 stalls should take half of the day. I know where every horse pees and poops, and which ones poop in their water. We use the wood pellets-- many people don't know how to use them. You put a bag in the stall, cut an x in the plastic and fill the bag with water. The pellets absorb the water and expand to like 5x the pellet size. You let it sit for 5-10 minutes, go do something else like grab hay. When you dump it out, it isn't sopping wet. Kick the new shavings around.
You don't have to have 5 inches or more (ugh) of shavings in each stall, they'll just take you forever to pick through, you will miss spots, and the horses will just make a pig sty out of it.
One trick is to pull about the front 1/3 of shavings away from the door and front of the stall where the feed buckets and water buckets are. That way you've got a clean/dry place for your hay to go, if the horse drops feed he can eat it off of the floor and not be eating shavings, and if he spills his water it can evaporate instead of wasting shavings. I use a broom.
Another thing I see when people are first learning to clean stalls, is that they take great big scoops of shavings/poo/urine and then rely on sifting sifting sifting for far too long. Why would you do that if you can pick up 80% waste and quickly sift the clean bedding and dump the waste. It goes faster if you use a wheelbarrow or a cart on wheels than carting crap with a tub. Don't carry shavings bag by bag, put all of what you'll need in a wheelbarrow and do them all at once, rather than completing one full stall (clean, bedded, watered, hayed, etc) before moving on to the next.
If your horses are burying all of their waste, you have too many shavings. There is no reason to take 1/2 day to clean 5 stalls... one stall can be completely finished in 10 minutes. Just automate everything. Have a routine. Get some small buckets and set up the grain for the next feeding in advance, stack them up and put a towel over the top so no insects get into it. Stack them in the order you'll walk to feed them, the top bucket being the first you feed, the bottom being the last.
You wouldn't think cleaning stalls could be such a science.