You have to respect that the barn is part of these people's home. While my barn does not have set hours (they realize some of us can't get away from work until 6PM), all the boarders respect that being there in the middle of the night is not fair to the owners.
Night feed is done usually after all the boarders have left (around 10PM). Morning feed is done around 5AM. Between those two times I don't think it's appropriate to show up to the barn, unless you are leaving for a show/clinic very early in the morning, or returning from a show/clinic very late.
They expect also that the last ones to leave will lock up and turn out the lights.
Barns that I have boarded at with hours have all been able to make exceptions for jobs, school hours, etc.. Mainly I think hours are mainly to deter inconsiderate behavior. Whenever I was at the barn "after hours" I would always help with final chores, sweep, help close up, etc..