Is his price real low? It sounds like you're getting into a horse that's fairly green and has issues. Try to talk the seller down in price too. With the rising costs of hay, grain, etc, horses are selling for a lot cheaper. Plus, it sounds like this owner doesn't really want to work with him, so I'm sure she'll lower the price.
Are you considering having him vet-checked? Maybe his frustration & random bucking is out of pain. Additionally, did you take a look at the saddle she's using? Maybe it doesn't fit right and is causing him pain. Especially since he's a Perch cross...maybe the saddle is too narrow.
When I first read your post, I thought, "OMG BUY HIM!" Just in your wording, you said all good things about the horse, and mostly bad things about his current owner (and how she works with the horse). I think it's okay if you've had 3 years off from riding. It sounds you might be a good match for this horse since you two can work from the ground up, as you said, and you don't have to jump right into advanced work.
A little side note: When I was considering to buy my horse, the owners said that he had frequently bucked them off. They were getting older and didn't want to endure the fall from a 16.1hh horse, so were selling him (plus other reasons). I've now had this horse for almost a year and not once has he ever attempted to buck or rear. A few times he has spooked at things which resulted in me hitting the dirt, but in no way has he ever bucked. So either, his previous owners were misinterpreting a spook or fresh horse as a buck OR my horse simply changed his behavior with a new rider/saddle/barn/arena/etc.
Whew! I'm typing a novel! But I vote go for it with this horse if you can get him for a good deal.
"He seems great to be around and I really love his temprament."