I'm only on my second horse, but, for me, a good gut feeling is the most important indicator of whether a particular horse is a good match for me.
Both of my horses have been the "crazy" ones, the kind that had been written off by their previous owners because they were too much. Both, for whatever reason, [apparently] acted completely differently for me that they did with their owners. Both times the owners pretty much stated from the get-go that the horse had chosen me and that I needed to take that horse.
I think, in horse buying, one should absolutely take calculated risks and avoid "[s]he's so pretty! I NEED him/her!" If the first thing you think when you see a horse is how beautiful your social media photos could be, that horse probably isn't the right one.
For instance, my first horse, my mare Lacey, was a pasture puff for 14 years before I got her because she had reared over backwards with a guy [at age 10] and seriously injured him.
She absolutely was a handful and she bluffed like she wanted to fight being ridden...but her goal was really just to scare her rider into dismounting and leaving her alone.
When she was ridden sensibly and fairly, and with an eye to her intelligence [she would try her heart out if she understood the "why" behind a request], she was a perfectly quiet mount.
The moment things started going sideways was the moment I, or anyone else, said "I know better than you, Lacey, so be quiet and get to work."
She turned into a fantastic lesson horse - she taught a lot of kids to love riding, and, 2 weeks before she passed away, I cantered her bareback and bridleless without any fear.
Plus it turned out, after I had owned her for 2 years, that she was mostly blind and had been for a significant portion of her life.
Before I met her, her previous owner had already made the appointment to have her put down, but then she saw me catch Lacey from the pasture. I have no idea what it was, but I remember walking Lacey over to her and hearing her whisper to my trainer "this girl is the one. Lacey chose her."
And it was true. I owned her for 4 years, never fell off once. We had more adventures, she saved my butt more than once, and she was always willing to do whatever I asked.
She was an absolutely wonderful creature and I got her because my gut feeling said so.
My current gelding is a "failed" therapy horse because he is "too spooky" for therapy work.
He was absolutely spooky as a therapy horse, but now, with me, without any desensitizing, he is 99% "cured" of spookiness.
It was the same thing with his previous owner - I was actually employed by her for a time and probably my second week working there, she came up to me and told me that she had had a dream that I owned Fabio.
I kind of thought that was ridiculous, but I also wanted it to be true. I had known from the moment I met him that there was "something" there.
Both of these horses gave me that positive vibe that you get from horses that are going to be good for you - kind of like how you can sometimes get a vibe from another human that the two of you are going to get along, before you even meet that person.
I think your reaction to the Dutch Harness Horse was the right one - it sounds like something put you on edge about him, and you listened to it. I think that's the key.
If something puts you on edge about a horse, listen to it, and probably run.