Moving freely on his own and moving while carrying weight on his back is different - so being comfortable with the former may not translate to comfort with the latter. Is your horse shod or bare? If bare, what sorts of footing is he being kept on, moving on on a routine basis (not just when he makes a break for it
As waresbear said, keep it at a walk - the concussive force of impacting on pavement at higher speeds is amplified on their structures - often with negative results.
Are you avoiding working him in the other areas due to your own concerns about the muddiness or is it that the BO has put restrictions on use of areas because of the wetness? I only ask because it is not necessarily automatically not possible to work in a wet area, as with working on concrete you would have to make appropriate adjustments to what you do to fit the conditions.