[trailscout]

Interesting, I’ve never heard of the Premack Principle before.

Well, now you've heard of him. Notice he predates Kurland's Loopy Training by a few decades or more. Be sure to check out Premack's background.

https://www.sas.upenn.edu/~premack/About.html

Dr. Patricia Barlow Iric in the video above expressed the opinion that she believes the Premack Principle is under recognized.

 

[trailscout]

For those that prefer academic discussions, here's an article by Dr. Jesus Rosales-Ruiz. I've read it a few time and it has cleared at least my mind of what exactly a poisoned cue is. Hardest part for me was remembering the acronyms used.


Edit: Oops. Just read where Ruiz's article had already been discussed. Oh well. I do also like Mary Hunter's explanations as well.

One more Edit please: When talking about poisoned cues, it is important to be using the +R definition of "cue". If the common concept of "cue" as used by -R trainers is used, the concept of poisoned cue has no meaning what-so-ever.

Poisoned cue only refers to a cue that has been conditioned through clicker/reward training.

If a horse has been taught to sidepass using clicker/reward, a cue of say a light touch on the shoulder with one finger can be established and the click can be faded out entirely. The light touch on the shoulder turns on endorphins etc to do the fun thing of sidepass. Introducing an aversive into this would clearly through a monkey wrench into the entire reward history. As in Huh? What?

I really appreciated Jesus's example that food can actually be turned into an aversive. As well as an aversive can be counter conditioned into an appetitive.

 

[loosie]

'Loopy training' - hehe, like that lable! I can think of quite a few loopy trainERS too! Yeah, I've never heard of 'loopy' in that respect, recognise the name Premack but forgot what his 'principle' was. One more case of be aware & be careful what you reinforce! So often people just don't realise what they are reinforcing or punishing.

One more Edit please: When talking about poisoned cues, it is important to be using the +R definition of "cue".

Yes, if you read what I wrote previously, we've established that some +R trainers think of 'cues' as exclusive to +R, and that's what 'poisoned cue' refers to.

Introducing an aversive into this would clearly through a monkey wrench into the entire reward history. As in Huh? What?

IF you're wanting to condition the cue to also be a reinforcer, it may confuse the issue, and it will weaken the cue as a reinforcer.

I really appreciated Jesus's example that food can actually be turned into an aversive. As well as an aversive can be counter conditioned into an appetitive.

I'm glad you understand that now. That's what I was trying to explain in the other thread. It is why I cringe when, for eg. my dog is at the vet having an unpleasant time & the vet nurse coos 'good doggie' at them.

 

[trailscout]

we've established that some +R trainers think of 'cues' as exclusive to +R, and that's what 'poisoned cue' refers to.

Yes, I did read that but it appeared to me some confusion still existed in that area. The article by Jesus that had already been posted and that I reposted clearly pointed that out. Jesus is not an animal trainer per se. He has a PhD in animal behavior. He is a behaviorist.

To be technical, or intricate, a cue is never a command irregardless of the fact that the term cue has come into incorrect popular use as a term referring to a command. A cue is an indication, not a command.

Ask-Tell-Demand is not and never was a cue. In direct and correct language, it is command, command!, command!!!

+R trainers and behaviorist with an educated background do not simply "think" of a cue as being only related to +R, that is in fact what it is.

The term cue as it is used in the pressure/relief world was adopted simply to sound nice as was the term "ask". The term command, which is what they are actually doing, just sounded too harsh.

Premack as it turns out has a contribution to animal psychology that rivals the work of Skinner. Not only did the Premack Principle (named that by others) come from his work, but also the notion of animal cognition and thought.

Where Skinner did his work in a laboratory setting, Premack took the classical conditioning and the four quadrants of operant conditioning into the wild to study how they operated in nature among many creatures large and small. He was the first to explore what was appetitive and aversive in nature and how those values could be changed and interchanged in nature.

Born in 1925, that makes him 95 today. He seems to be the typical reclusive scientist that does not want or desire a lot of public recognition but is happy to live and work within his peer group.

Any student who studies animal behavior in an academic setting will know all about Premack's contribution to animal psychology. I'm sure he was mentioned in my psychology courses 50 years ago. I suspect he was sidelined because of his belief in animal thought and cognition which was not popular at the time but which has since become solidly established.

Ran across this definition of poisoned cues:

"So what is a poisoned cue? Technically speaking it is an antecedent that has been paired with both positive reinforcement and punishment (or correction), and begins to elicit undesirable emotional responding when in the presence of the performer. "

Looking further into the history of poisoned cues found this: "Karen Pryor coined the term “poisoned cue”. She commented that behavior analysts can easily set up experiments that look at pure positive or pure negative reinforcement. But Karen said that’s not the real world. The real world is filled with mixed consequences. What happens when a cue can lead to good things happening or to bad things happening but you don’t know which it is going to be? "

Although Karen Pryor is said to have coined the term, the background of real world consequences was first studied and researched in great depth with a mound of contributions by Dr. Premack all so many years ago.

Here's another definition of poisoned cue that to me is noteworthy:

"A poisoned cue means that when the cue is presented, the animal can earn reinforcement if it does the behavior correctly OR it can expect some kind of aversive if it does not perform the behavior. Because the cue is no longer just an indicator that something good could happen, the cue itself becomes ambivalent. "

The audio course offered by Mary Hunter and Alexandra Kurland has 6 units. Unit number 5 is: Quote: The Premack principle gives you a different way to understand reinforcement and a new way to find reinforcers. Our conversations will help you better understand this principle so that you can use it in your training.

Listen and Learn Audio Course

Mary Hunter has teamed up with Alexandra Kurland and Dominique Day to create a 31-part audio course that explores the connections between applied behavior analysis and animal training. This course explores the basic principles of behavior analysis. However, this isn't your typical online

stalecheerios.com

All this talking is a good thing, at least for me, as it stimulates me to dig faster, further, and deeper that I might otherwise.

 

[JoBlueQuarter]

This is starting to make a lot more sense to me, as I keep it in mind when I work with my horses and am starting to recognize the difference. I think the "combination" of +R and -R works (and I'm not talking about the creation of a poison cue, so much) when we have enough of a relationship of trust with our horses that they are alright with things not always being their choice (could be argued that they prefer it) simply because they trust their handlers enough. And how good the handler's are at "reading" their horses and thus balancing it well. But I'm a lot more careful using aversives now that I realize the problem it causes for Heidi.
My next confusion is how this can relate into ridden work. I suppose, the same methods can be applied just in a different sense and less food rewarding (compared to what I do now). How have you balanced this? What core principles did you take with you into ridden work? Loosie, you haven't been working "cleanly" +R so far, right? So how have you trained the ridden work? What balance did you strike, between being very commanding and only "requesting"? What are your guidelines and what have your experiences been... I'm here to pick y'alls minds. Mainly I go with my gut... But it would be helpful to have an idea of how you guys have accomplished this. It's easy, for me, to work like this while on the ground, but I do have trouble relating it to ridden work. However I don't believe that ridden work has to be done in the normal "negative" way where the horse is constantly being forced again (I think this part is a bit of a grey area - at least when it comes to my knowledge - but I would like to make it clearer). At least I've built a foundation so strong with Heidi that she ain't gonna be "broke" now, by anyone! 😄

 

[trailscout]

I think the "combination" of +R and -R works

Curious what you mean by combination.

My horse was trained with -R/pressure release and that is how I have always ridden him. But now I'm introducing +R activities. The thing I have to be careful about is to not use -R and +R for the same behavior. This is what confuses the horse and creates poisoned cues. Once +R is introduced to a behavior, that behavior must remain pure +R. Another unrelated behavior can be used with -R with no danger of creating a poisoned cue as long as +R is not introduce to that behavior.

The -R may of course interfere with the relationship +R is building.

Again, poisoned cue only relates to a cue that was taught using +R. I think maybe I'll start using -cue and +cue to avoid the confusion and not tick people off by insisting on using command for -cue.

+R behaviors are introduced to -R trained horses all the time with no adverse effects.

 

[JoBlueQuarter]

Curious what you mean by combination.

My horse was trained with -R/pressure release and that is how I have always ridden him. But now I'm introducing +R activities. The thing I have to be careful about is to not use -R and +R for the same behavior. This is what confuses the horse and creates poisoned cues. Once +R is introduced to a behavior, that behavior must remain pure +R. Another unrelated behavior can be used with -R with no danger of creating a poisoned cue as long as +R is not introduce to that behavior.

The -R may of course interfere with the relationship +R is building.

Again, poisoned cue only relates to a cue that was taught using +R. I think maybe I'll start using -cue and +cue to avoid the confusion and not tick people off by insisting on using command for -cue.

+R behaviors are introduced to -R trained horses all the time with no adverse effects.

I'm still trying to figure out what I mean with combination lol. I guess all I mean is using both in the overall education of the horse, but like you said not on the same behaviours? OK so in the training of a -R behaviour, you must stick purely with -R? What does this mean, in effect. You use pressure and release to "tell" the horse to do something and when they do it must you release pressure but only release pressure? Not add verbal praise or patting, etc? Would praising the horse "poison" it? Or this is not where the separation must lie. You can praise your horse and reward them but a "-cue" must always be backed up ("forced", if you will). The only difference is that with a +cue you ask but are fine with not getting a response whereas in -cue you ask (or tell, if you prefer) with the intent to back up until you get your response.
I'm trying to figure out how much -R I can/should use, without causing bad consequences, and how to keep from inadvertently poisoning cues.

 

[trailscout]

A -cue cannot be poisoned. Period. Has no meaning. Karen Pryor coined the term and it only applies to +cue.

When you apply a -cue to stop, say lifting the reins or back a little, the horse has learned to stop to escape the pressure and avoid the addition of more pressure.

If you praise him for escaping and avoiding the pressure you applied with the -cue, he may appreciate your happy voice and will enjoy a treat if offered, but he will not connect it with escaping/avoiding the pressure.

Well, unless you fed or praised while the -cue was being applied or even right right after in which case he could associate the praise with the aversive pressure. If done correctly, or incorrectly enough, the "good boy" could become a -cue to stop. We wouldn't want him to stop every time we said good boy.

That's how targets are replaced by new cues.

 

[untamed equestrian]

'Loopy training' - hehe, like that lable! I can think of quite a few loopy trainERS too! Yeah, I've never heard of 'loopy' in that respect, recognise the name Premack but forgot what his 'principle' was. One more case of be aware & be careful what you reinforce! So often people just don't realise what they are reinforcing or punishing.

Haha yes! Defintely agree with you, antecedent arrangement and what comes before the cued or desired behavior is so important to keep in mind when training - otherwise you can get an undesired “loop” in your behavior.

 

[trailscout]

Something sort of dawned on me today. Keno was trained using -R and then ridden by me using -R -cues. And somewhat frequently there was a "good boy" for compliance. Which really didn't mean anything to him I'm learning, but it seems there could have been an association of some kind formed between "good boy" and aversives.

I thought about this today as I was paring "good boy" with the click.

This doesn't fit the idea of poisoned cues, but it does sound like it could cause some confusion in his head. So I've decided when he is being -cue for a -R behavior, no more "good boy", just in case.

Has anybody heard/read any discussion pertaining directly to this?

 

[untamed equestrian]

Has anybody heard/read any discussion pertaining directly to this?

If Keno is happy and content when you tell him ‘good boy’ I would think it’s fine! Even in -R when you tell a horse good boy you are usually happy as compared to frustration or aggression when you hit your horse with a crop because he keeps ducking out of a fence. I believe that horses can sense our energy and read a lot of our body language, and good boy is often connected with the horse being released from the pressure, given a scratch, or something like that. However, I think things can absolutely be linked to an aversive, negative emotions, or a bad experience.

 

[loosie]

Meant to get back to this - started writing earlier reply, still sitting unfinished, been busy... Also meant to look up Premack again, esp interested, as you said he was into ethological studies, Trailscout. But for now, just saw this...

could have been an association of some kind formed between "good boy" and aversives. ...
This doesn't fit the idea of poisoned cues,

Yeah, same-same, as to whether something can be associated to a 'good' or a 'bad' consequence. In vast majority of horse training(& you say Keno's), 'good boy' is mostly associated with cessation of aversives(-R). So... if you're going to start using it as a cue/marker for a +R then you could say it does indeed fit the 'poisoned cue' idea, just in reverse - could call it a sweetened cue ;-)

As the vast majority of horses I've dealt with have previously been trained by someone else, that is precisely what happens, if the word 'Good!'(or whatever cue/signal) was associated solely or mostly with -R - they then come to me and find it also often means +R. It's not something I think they are confused by at all - it generally starts out as a nice surprise that they get a 'lolly' or a good scratch or such, then Good Stuff becomes anticipated, something worth trying for.

 

[trailscout]

when you tell a horse good boy you are usually happy

This is something I've believed in long before I learned about +R. It sounds silly to some to talk to your horse (or dog) because they can't understand a word we say (unless it's been paired as a cue). But they understand our emotions and our emotions to a large degree follow our words.

Maddy gave an example of a study where one set of people were holding a pencil in their teeth which produces a smile while the other group held it in their lips which produces a frown on the lips. I forget the detail at the moment but the scoring on pictures or what ever test followed were far different for the two groups. So yeah, happy body language and face are as important as our words sometimes.

Shawna is constantly reminding her listeners of the importance of both the trainer and horse enjoying the training sessions.

So... if you're going to start using it as a cue/marker for a +R then you could say it does indeed fit the 'poisoned cue' idea, just in reverse - could call it a sweetened cue ;-)

Yeah, that's on the order of what I was thinking. Not as just an random 'good boy' but actually paring it with the clicker so as to use it as a bridge under saddle with a wither scratch or treat coming later.

It was like, I think I'll start saying good boy with the click to see if it can get loaded as a click as well. Then as I was doing it I thought, wait a minute, I do this a lot with -R.

So with -R and +R activating such different parts of the brain, it left me wondering. Perhaps better to pair it with a mouth click.

And as I mentioned, Kurland and Hunter devote 1/6 of their on line course to the Premack Principle with a total of 12 hours of discussion just on the Premack Principle. So it must have a few intricacies of it's own!

 

[trailscout]

antecedent arrangement and what comes before the cued or desired behavior is so important to keep in mind when training

In re-reading Shawna's first book I thought about your post when she told a story about training a horse to do a spin from the ground. When her husband tried the cue the horse just looked at him.

So they examined what she was doing and she was making a shift in the direction of the spin along with the cue. So she faded that part and all was well with the cue alone.

Temple Grandin's books provide a lot of detail how absolutely specific everything is for animals, an area she has personal knowledge of.

 

[untamed equestrian]

In re-reading Shawna's first book I thought about your post when she told a story about training a horse to do a spin from the ground. When her husband tried the cue the horse just looked at him.

So they examined what she was doing and she was making a shift in the direction of the spin along with the cue. So she faded that part and all was well with the cue alone.

Horses pick up on really subtle shifts in our behaviors and facial expression. Reminds me of Mosie Trewhitt, her cues are so subtle that you have to really pay attention to catch them, yet her horses never miss them.

Temple Grandin's books provide a lot of detail how absolutely specific everything is for animals, an area she has personal knowledge of.

I read one of her books a while ago... might go back and read it again. It really makes you see just how complex the animal mind is.

 

[trailscout]

I read one of her books a while ago... might go back and read it again.

I have two of her books. She talks about thinking in pictures and believes animals do also. And I remember her saying if she saw a church or barn, forget which going in one direction it was a totally different building from the other direction. And detail. Yes and lots of it, maybe all of it.

I go back and re-read stuff and look up stuff occasionally. The most fantastic thing I read was about a macaw or similar that actually learned to spell a word without being taught anything other than the sounds of the letters of the alphabet. She was frustrated for not getting the usual treat rewards because the trainer was focused on the person that was furnishing the money and was visiting. I forget the birds name, I'll say Polly. She could talk and say nut but she could not spell it and had never been taught. Polly want a nut........Polly want a nut.............Polly want a nut...N-U-T!

The trainer was floored as she was only demonstrating how well Polly knew the sounds of the letters. Polly figured it out and wanted her nut!

Oh well, so much for the bird brain theory.

 

[untamed equestrian]

I have two of her books. She talks about thinking in pictures and believes animals do also. And I remember her saying if she saw a church or barn, forget which going in one direction it was a totally different building from the other direction. And detail. Yes and lots of it, maybe all of it.

I go back and re-read stuff and look up stuff occasionally. The most fantastic thing I read was about a macaw or similar that actually learned to spell a word without being taught anything other than the sounds of the letters of the alphabet. She was frustrated for not getting the usual treat rewards because the trainer was focused on the person that was furnishing the money and was visiting. I forget the birds name, I'll say Polly. She could talk and say nut but she could not spell it and had never been taught. Polly want a nut........Polly want a nut.............Polly want a nut...N-U-T!

The trainer was floored as she was only demonstrating how well Polly knew the sounds of the letters. Polly figured it out and wanted her nut!

Oh well, so much for the bird brain theory.

Yes, I remember reading that! I think it was a barn, she was saying how animals see all the details of the object or thing before seeing the big picture and that they don’t generalize things well. I had never heard of that before, and I thought it was really interesting and made a lot of sense.
I’ve read a story similar to that of Alex the African Grey parrot, his owner was busy so he sounded out the word to get her attention...parrots are so incredibly smart. I actually have one and parrot behavior has been an interest of mine for awhile. I think that they are one of the smartest animals.

 

[trailscout]

I looked up Alex. He is billed as the only non-human animal to ever ask a question with one of the questions being an existential question. Just wow!

I have heard people incorrectly claim that a horse's brain was no larger than a walnut. Sounds like that wouldn't be a great handicap even if it was true.

This African Gray Parrot Was the First Animal To Ever Ask an Existential Question

This "bird brain" has helped scientists realize that birds are actually quite intelligent.

mymodernmet.com

 

[trailscout]

My question is about where the individuality of the horse comes in. Meaning, isn't it important to figure out if our reward is truly a reward or a punishment?

I

I would like to take a shot at addressing this.

I have been listening to podcasts by Shawna Karresch starting with #1 from about 2 1/2 years ago.

She talks about the individuality and individual quirks of every horse she works with. During the first two episodes she explained that when a horse was of the more excitable type it was important that the sessions for not mugging only be attempted after a turn out when most of the energy was gone, or at least some as some never run out it seems.

On the other hand, she explains that when a horse is so shut down as to not want to interact with humans at all, the procedure is to use a higher value reward and even try to get the horse to mug you. As in please mug me! When the horse finally wishes to at least mug you a little consistently with some trust, then the exercise with the clicker to tell the horse to hold his head straight forward can begin.

In the end, +R is the same ole same ole. But in the lead up, yes, individuality does have to be recognize. But even then, the same ole same ole will need to be used creatively for some differences. I'm jut learning.

And about "isn't it important to figure out if our reward is truly a reward or a punishment?"

A reward is something that comes after a behavior that increases the chances of the behavior occurring again.

Punishment is something that happens after a behavior that decreases the chances of the behavior occurring again.

So yeah, it's important to know what is punishment to the horse. But it's important to recognize that punishment is something done to the horse or that the horse is made to do in terms of +P.

Standing still is very hard for some horses. Shawna works mostly with Thoroughbreds and hi energy jumpers at that. They are trained to stand still using +R. And of course some trainers do the same training but with -R.

But when a horse is allowed to rest at the end of a round pen session, the horse is not usually being MADE to be still but rather ALLOWED to be still. At that point the horse really WANTS to stop for a while. And because the horse wants to stop some people incorrectly conclude that it's rewarding to the horse. And in one sense of the word rewarding, it may be, but not in the sense that it will cause the horse to want to voluntarily run around in the round pen again.

I remember in one podcast Shawna quipped, "the horse is not running around saying would someone please put some pressure on me so I can experience the release or rest, but rather saying 'can i get some grass".

I don't know how much you know about Shawna Karresch, but she worked full time as a trainer in Sea World for ten years. Full time job. She said it could take as long as a year to train a sea lion to do a back flip as it was not a natural movement. No pressure release used on sea lions or orcas.

She became interested in horses 25 years ago and left Sea World. One of the first horses she worked with was ridden by Beezie Madden who took home a single jackpot of a cool one million dollars.

Since then she has worked with thousands of horses and currently works at a training facility with forty stalls. She does clinics and has some books but mostly she is a hands on full time +R equine trainer. I doubt there is any with her hands on experience in +R equine training. Her personal horse was trained 25 years ago with 100% +R. If it can be done with orcas and sea lion, there is no reason it can't be done with horses.

The big hitch is that initially, +R just takes a bit longer than -R. But at some point on the advancement curve, I at least am beginning to believe that the time factor turns in favor of +R.

 

[gottatrot]

I

I would like to take a shot at addressing this.

I have been listening to podcasts by Shawna Karresch starting with #1 from about 2 1/2 years ago.

She talks about the individuality and individual quirks of every horse she works with. During the first two episodes she explained that when a horse was of the more excitable type it was important that the sessions for not mugging only be attempted after a turn out when most of the energy was gone, or at least some as some never run out it seems.

On the other hand, she explains that when a horse is so shut down as to not want to interact with humans at all, the procedure is to use a higher value reward and even try to get the horse to mug you. As in please mug me! When the horse finally wishes to at least mug you a little consistently with some trust, then the exercise with the clicker to tell the horse to hold his head straight forward can begin.

In the end, +R is the same ole same ole. But in the lead up, yes, individuality does have to be recognize. But even then, the same ole same ole will need to be used creatively for some differences. I'm jut learning.

And about "isn't it important to figure out if our reward is truly a reward or a punishment?"

A reward is something that comes after a behavior that increases the chances of the behavior occurring again.

Punishment is something that happens after a behavior that decreases the chances of the behavior occurring again.

So yeah, it's important to know what is punishment to the horse. But it's important to recognize that punishment is something done to the horse or that the horse is made to do in terms of +P.

Standing still is very hard for some horses. Shawna works mostly with Thoroughbreds and hi energy jumpers at that. They are trained to stand still using +R. And of course some trainers do the same training but with -R.

But when a horse is allowed to rest at the end of a round pen session, the horse is not usually being MADE to be still but rather ALLOWED to be still. At that point the horse really WANTS to stop for a while. And because the horse wants to stop some people incorrectly conclude that it's rewarding to the horse. And in one sense of the word rewarding, it may be, but not in the sense that it will cause the horse to want to voluntarily run around in the round pen again.

I remember in one podcast Shawna quipped, "the horse is not running around saying would someone please put some pressure on me so I can experience the release or rest, but rather saying 'can i get some grass".

I don't know how much you know about Shawna Karresch, but she worked full time as a trainer in Sea World for ten years. Full time job. She said it could take as long as a year to train a sea lion to do a back flip as it was not a natural movement. No pressure release used on sea lions or orcas.

She became interested in horses 25 years ago and left Sea World. One of the first horses she worked with was ridden by Beezie Madden who took home a single jackpot of a cool one million dollars.

Since then she has worked with thousands of horses and currently works at a training facility with forty stalls. She does clinics and has some books but mostly she is a hands on full time +R equine trainer. I doubt there is any with her hands on experience in +R equine training. Her personal horse was trained 25 years ago with 100% +R. If it can be done with orcas and sea lion, there is no reason it can't be done with horses.

The big hitch is that initially, +R just takes a bit longer than -R. But at some point on the advancement curve, I at least am beginning to believe that the time factor turns in favor of +R.

Very interesting. Great post.