You'll have to check with the bylaws of the town. For my town we are allowed but you must follow bike rules and clean up after the horse. It differs place to place, but if you find out that you indeed can, I'd recommend bringing along a letter from your officials or a copy of the written law stating that it is legal because some people surprisingly get defensive when they see horses on the road.
We are not human beings having a spiritual experience. We are spiritual beings having a human experience. -Pierre Teilhard de Chardin