Well, this is probably going to sound a bit "snobby",
But it's not meant to be...
I think it depends upon the value of the horse.
If you are looking to spend tens of thousands on a well-bred show horse that you must go to special breeders to find? Then distance should not be an object. And with that kind of purchase, you should be willing to pay travel expenses for whomever is accompanying you to evaluate the horse.
If you are spending just a few hundred dollars? I would think you'd be able to find plenty of good horses close to home...no need to travel.
And if your purchase prospect is somewhere in between those two extremes? Then do what seems reasonable to you...
No need to break the bank on travel expenses if it's going to cost you more than the horse!
Leasing a spoiled rotten trail horse...pretty - but a brat!