I'd say don't give yourself too much of a headache over getting him totally dust-free. Unless you're showing, there's really no need, and he's going to be dirty/dusty again in a few hours. Barns are just dusty environments-it settles on everything-and horses are going to roll, lie down in shavings, etc. and its going to all get worked into their coat. When I bathe my mare (and I mean like hard-core show-time bubble bath) she still sometimes manages to kick up dirt or dust after she dries. It's crazy. Ultimately its all part of a losing battle we humans seem determined to engage in
and I usually leave the intense grooming to show season. The most important thing is that he's clean under any harness or gear you might put on him.
As far as those "magnetic spots" go, I've found that nothing beats a lot of elbow grease with a curry comb and dandy brush. The curry really stirs up whats under the hair and the dandy brush is there to flick it away-you'll have to be prepared to do several rounds of this. I don't know if you've been using fly spray, but sometimes the oil-based ones will attract more dirt/dust to the coat. Body clipping him might help, but that's obviously not something you would want to unless you had other reasons.
Good luck with your little guy-minis are fun. I couldn't help chuckling at this post because I used to drive a mini named Dusty ;)