There are a handful of ponies who compete in my region, mostly ridden by juniors, but there is at least one woman I have seen riding one as well.
I think their stride length would have a lot to do with their ability on trail. A really small, short-strided pony would have to take twice as many strides on trail as a bigger animal, so would be working much harder. Doesn't mean they couldn't do it, just another thing to keep in mind while training.
But considering a lot of arabs are pony-sized (a friend who has won several rides this year, including the OD 50, rides an arab who is 14.1 with shoes), there is certainly no reason a pony can't do endurance and do it well!
There is no joy equal to that found on the back of a horse.