Like most people here, it's usually much more clear when you know a horse is NOT the right one. When I was horse hunting again, I can't even count all the horses I went to look at, and each one was a completely different experience. Here's some bad ones I've had:
1) Was a beautiful draft, very well trained. Good experience with the horse and the owners, but when I rode him, he just had too much forward momentum I felt unsafe on him. He was the biggest horse I'd ever seen in person, and I'm a short 5'2". I needed a stool just to brush him. I mean... he was lovely. But I knew it wouldn't work out.
2) A cute paint. I was already uneasy when I got there because he was already tacked up when I arrived. That's a pretty big red flag. I asked the owner to get on first, and he bucked and crowhopped and had a really choppy trot. I still got on him anyways but I knew it wasn't right. Never went back.
3) Another draft, a beautiful one. I liked the owners a lot, but again, when I arrived she was already tacked up and someone was getting on her. ALL SHE DID WAS WALK AROUND THE ARENA. And the owner wanted me to buy her RIGHT THEN AND THERE. Literally. She didn't even trot or canter. She walked her around once and the dude was like...asking me to buy her. I was thinking is this guy serious? He didn't even want to let me ride her, but still wanted me to buy her. Also, I was 90% sure the horse was drugged from the way it was acting and that's probably why she only walked her. I got out of there pretty quickly.
And then finally, I found the right boy for me. And I only tried him once, so I definitely think it's possible to buy the right horse only riding them once.
This time, the owner walked with me to get the horse and let ME lead him, groom him, and tack him up. That already was a good sign. She got on him first, and rode him correctly showing off his gaits and his skills (Leg yielding and sidepassing, etc.) Showed me how he stands still for mounting, etc. Then I got on him and rode him. She told me about his history, and other things about him. I already knew he was for me. Then I was able to untack him and put him back in his pasture. The owner did everything right in helping my decision to buy this horse, and I ended up buying him, so I'd say it was a success :)