You would probably need to stick to the horse trails instead of actually hiking on hiking trails. Will you be trailering the horse? The terrain around my boarding facility is all hill. You're either going up a hill or down a hill. There are some patches with smooth, even ground but the majority of it, you are either ascending or descending.
I've only trail ridden once or twice on public tails and I don't recall much about it. I'm pretty sure that if you decide to hike in a park, you would have to stick to the equine trails since you'll have a horse, and horses tend to poo. Just like some parks will only allow dogs on dog trails. Normal hikers don't want to have to share the trails with animals.
I would take into consideration your location. I live in western PA. We are at the foothills and you rarely have a long stretch without some sort of incline. I couldn't imagine hiking these trails (around my barn anyways). It would just be rather unpleasant. Even the "mild" trails at the local parks have hilly climbs. Would the horse trails be very hilly, or would they be flat? How long would you be out? Ontop of everything you'll pack the mini with, you'd have to take into consideration how to get the horse to the trail. What will the mini do during the winter months, when you'll hike less (I'm presuming)?
Overall, I think it would be a fun idea. You just have some things to think about.