I would look at appaloosas, QH, paints, or any of the above crossed with arabs. I am not as familiar with other breeds as I am with apps, but I know that most of the ones I've had can go all day.
We had a stud horse that worked 8-12 hours a day on the farm and had no soundness issues. (I'm sure this is true of many other breeds, too.) One of my favorite things about him was that I could pull him and a couple of the mares out of the pasture, throw on some saddles, and go for a relaxing trail ride. Some of that was training, but most of it was disposition.
There are horses in every breed that are harder to handle than others. I wish you luck in finding the right horse for you!
Some thoroughbreds can also make good endurance horses the lady I used to lease from had a younger throughbred mare who excelled in endurance. But I agree with everyone arabians and arabian crosses are excellent endurance horses as well as great kids horses if they are trained properly I knew 2 arbian geldings who took many kids to their first gymkhanas over their first jumps and to their first canter/trot they are awsome versatile horses
My POA still rivals the younglings on long, taxing rides and she's 32. I have yet to see a horse makea distance in as short of a time as she can. Most POAs I've met live long lives with long riding careers. I used to ride endurance on my Polish Arab and she did really well too. Though neither of those ladies would be what you called of a refined build. Both were quite stocky for little gals (13.2 and 14hh).
I'd say your best bet is to get out there and see whats in your price range that fits what you want as a riding horse.