Clicker Training: Challenge Accepted - Page 2
 
 

       The Horse Forum > Training Horses > Horse Training

Clicker Training: Challenge Accepted

This is a discussion on Clicker Training: Challenge Accepted within the Horse Training forums, part of the Training Horses category
  • Variable ratio schedule clicker training horses

Like Tree297Likes

 
LinkBack Thread Tools
    02-13-2013, 01:51 PM
  #11
Yearling
Reinforcement: The nitty gritty details

In order to understand clicker training, you must understand the terms that apply to behavioral training in general: positive, negative, reinforcement, and punishment. These govern ALL motivated behavior, whether we're talking about horses, lizards, birds, bugs.... Everything. Even you humans reading this.

Here's a little upper-division psychology for you :)

First, lets define them:
Positive: this is like a positive in math, not positive meaning "good". It means adding something that wasn't there before.
Negative: this is also like math - a negative number means you're taking something away or subtracting it. This is not "bad", so don't confuse this definition of negative with one that is judgemental in nature.
Reinforcement: this is anything that causes a behavior to happen more often, as in it reinforces the behavior.
Punishment: this is anything that causes a behavior to happen less often

**Note: something can be intended to be a reinforcer or punishment that is NOT actually a reinforcer or punishment since it is ineffective and does not achieve making the behavior happen more or less often. I addition, something that was a reinforcer or punishment can stop and start being either. It all depends on whether it is currently effectie to change behavior or not.

Now, let's combine them and discuss the results:

Positive reinforcement: by definition, this is adding something that causes the behavior to happen more often. For example, when you work, you receive money, so you work more to receive more money.

Negative reinforcement: again, by definition, this is taking away something that in turn causes a behavior to happen more often. For example, there is a sqeaky door that really annoys you. However, when you put WD-40 on it, it stops sqeaking. This leads to you putting WD-40 on the door whenever it sqeaks so that the noise goes away. Taking the squeak away is negatively renforcing you putting the WD-40 on the door!

Positive punishment: this is when something is added that causes a behavior to happen less often. For example, when a child misbehaves, spankings are added to the scenario. So, the child misbehaves less often.

Negative punishment: this is when something is taken away that results in the behavior happening less often. Say a teen stays out too late, so driving privilages are taken away. Afterward, the teen does not disobey their curfew again.

Now that I've clarified what these terms mean, I'll apply each to horses in my next post. For now, I need to grade some papers :)
rookie and PunksTank like this.
     
Sponsored Links
Advertisement
 
    02-13-2013, 01:55 PM
  #12
Yearling
Quote:
Originally Posted by jillybean19    
For now, I need to grade some papers :)
By the way, I need to grade papers because:

If I do, my students will be happy to find out their grades (positive reinforcement)

If I do, I can go home earlier and not have to stay at school (negative reinforcement)

If I don't, I risk hearing about it from my boss (positive punishment)

If I don't, I risk loosing my job (negative punishment)

There's some hard-core behavioral operant conditioning for you - ponder that one for a moment!
PunksTank and jmike like this.
     
    02-13-2013, 09:38 PM
  #13
Yearling
Operant Conditioning: Applying it to horses

Ok, so in my last post, I explained the different aspects of operant conditioning. (Operant conditioning is simply a fancy psychology term for saying we can train behavior through motivation and is applicable to pretty much every voluntary behavior known to living organisms).

Now, let's apply it to horses:

Positive reinforcement: This is where clicker training finds a home. With clicker training, a reinforcer (usually a treat) is introduced when the desired behavior is performed. Technically, treat training is also positive reinforcement as far as it is able to reinforce the behavior you're wanting. The addition of a clicker as a "bridge" between the the behavior and actually receiving a treat simply allows us to be more intentional, accurate, and flexible with the behaviors we are trying to reinforce, which I already discussed in a previous post.

Negative reinforcement: This is where training off of pressure comes into play. In "traditional" training, pressure is applied to guide/ask the horse to do something, and then the pressure is released when the horse responds correctly. The horse learns to work for the release of pressure.

Positive punishment: This is used each time you smack your horse for getting in your space. For instance, if he is mugging you for treats and you give him a firm thwack on the nose, he learns not to mug you for treats or else!

Negative punishment: I had a hard time coming up with one for this, but it just dawned on me yesterday - this is often used in *proper* clicker training if you are going to give the horse a treat and the horse reaches out for it a little too eagerly. The correct thing to do in this situation would be to wrap your fingers around the treat and take it back. Withholding the treat discourages the horse from reaching for it, and since you are eliminating that behavior by taking something away that he would have gotten otherwise, it is negative punishment.

Oftentimes, these are simultaneously used to govern behavior. I already gave an example above about how each of these applied to why I graded those papers, but now here's an example of combining these within a training session: If I wanted to teach my horse to back up (as I did yesterday), I first gave the cue that I want him to learn ("back up"). However, since that didn't mean anything to him yet, I stepped forward and put pressure on his front shoulder. He took a step back, and I released the pressure (negative reinforcement), clicked, and treated (positive reinforcement). However, if he ever reached for a treat, I would have bopped him on the nose (positive punishment) and withheld the treat (negative punishment). During any given training session, you'll usually find me using a combination of positive and negative reinforcement (clicking and treating as well as using pressure). I don't use punishment unless he does something I don't want, obviously. Usually, that doesn't happen, though, since he's so keen on trying to figure out what I want :)

There are many other examples of how these are used with horses and in our everyday lives. Hopefully, you're beginning to get an idea of how important these principles are and how they apply to just about everything you do. For example, I am currently writing this because I want the positive reinforcement of hearing other people's comments and knowing I helped them learn something as well as the hoped-for negative reinforcement of fewer people writing off clicker training simply because they don't understand it.

Which, by the way, brings us to reinforcement schedules and how some schedules are more powerful than others in sustaining behavior - but that's for another post ;)
PunksTank likes this.
     
    02-13-2013, 11:41 PM
  #14
Yearling
Reinforcement Schedules

Once you understand reinforcement and punishment and the different ways that they work, now it comes to WHEN you reinforce. This isn't quite as critical as knowing why training works the way it does in the first place, so I'll keep it short and sweet for those that are interested in the various types of reinforcement schedules and the results they produce.

The first main type of reinforcement schedule is continuous reinforcement. This is the type of schedule most commonly used in clicker training. Basically, this means that the behavior is reinforced each time it's given. In other words, if I'm teaching my horse to pick up his feet, I click and treat each time the horse picks his foot up. This is best used during the initial stages of learning as it create a strong association between the behavior and reinforcement. However, once the behavior is firmly associated with the reinforcement (and your horse knows what you expect from him), you can do two things - ask for more and/or switch to a partial reinforcement schedule. Personally, I do both. I'll explain the "asking for more" in the next post, but the basic idea is that the horse has to take the behavior one step further before getting a reinforcement (now he has to hold his foot up longer... and longer....) and you're actually asking for your horse to learn something new (i.e. Holding a foot rather than just picking it up). However, for this post, I'm going to explain switching to a partial reinforcement schedule in order to reinforce the SAME behavior that was already taught. This prevents what we call "extinction" - in other words, the behavior stopping since we're not reinforcing it anymore (for those of you who think that a clicker trained horse will ALWAYS need a clicker, listen up!).

Partial reinforcement: this means that the horse doesn't get a reinforcement every time it does what you're asking. Instead, it only gets a reinforcement part of the time. This way, you can ask for the behavior more often without a reinforcement (i.e. You can ask for behavior without a clicker) and the horse will still respond even though it doesn't get a treat every time.

I'm only going to worry about the things that are most important here - if you want to know more, Google "reinforcement schedules".

Here are the key terms you need to know to understand partial reinforcement:
Fixed = when you reinforce doesn't change.
Variable = it's unpredictable when you'll reinforce behavior
Ratio = when you reinforce depends on the number of times the behavior is performed
Interval = when you reinforce depends on the amount of time that has passed (I'm not going to discuss this one here, though).

There are four types of partial reinforcement, and different schedules lead to different results. I've included a graph below that illustrates these. I'm only going to explain fixed ratio and variable ratio here, though, because they directly apply to clicker training.

Fixed ratio means that you reinforce after a specific number of correct behaviors. Generally, this leads to a steady rate of responses in order to earn the reward with only a brief pause after getting the reward. For example, every time a kid completes three math problems, he gets a piece of candy, so he does three math problems, receives his candy, eats it and pauses, then decides he wants another one so gets back to work again. The weakness here is that, if the horse doesn't receive a treat after the expected time, the behavior can break down and the horse stops responding.

Variable ratio solves this problem. With variable ratio reinforcement, the horse never knows when it's going to get a reward - it can perform the desired behavior any number of times and may or may not receive the reinforcer. This is the most powerful reinforcement schedule as it produces a high and steady rate of the desired behavior. Don't believe me? This is how gambling addiction works: You never now when you're going to win, even without any sort of reward (and even lose money!), people keep on gambling and gambling because every now and then they win $5 back, $2 back, $10 back, etc., and they think they just might hit the jackpot with the next round.

This applies to clicker training when teaching the horse to respond the way you want it to without the clicker. I use this to reinforce behaviors that my horse knows and that I expect, but want to reward every now and then. When I'm "phasing out" the clicker, I'll ask the horse to do what I want and only click and treat every now and then. Thus, he learns that he can respond even without the clicker. Eventually, I won't use the clicker at all when asking for this behavior - this behavior is expected and the horse knows what he's supposed to be doing (thus I avoid the horse trying something else or getting confused because he didn't get a reinforcing click and treat). For things my horse knows REALLY well, I do click and treat every now and then just to say "good boy" in a way that's meaningful for him. I could probably be just fine without it, but I like to reinforce these behaviors every once in a while (i.e. Maybe once in a week or even a month) just because. Since he never knows when what he's doing might earn him a treat, he's always listening even when he doesn't get one!
Skyseternalangel likes this.
     
    02-14-2013, 12:05 AM
  #15
Weanling
I am greatly enjoying reading these posts... It must be a lot of work, but I am very interested in the process, progress, and imput!
jillybean19 likes this.
     
    02-14-2013, 12:11 AM
  #16
Yearling
Pats and Verbal Rewards: Are they reinforcement?

I have never been able to wrap my mind around WHY we seem to think that patting a horse or telling it "Good Boy" would be rewarding for a horse. Personally, I think we do it because we find it rewarding. Human language on its own is meaningless sto a horse, and I can't imagine that the horse (or any animal) really wants to be patted - with one of my horses, it would actually be counter-productive since one of my horses is really sensitive to things like that and shies away from them.

Realistically, the only way a pat or a verbal reward could be any sort of reinforcer would be if it was done consistently enough with other things to become associated with those things. For instance, if your horse gets a quick break or a change in activity when they get their pat or "good boy", they could become associated with one another. Essentially, you've done the same thing that clicker training does when it creates a "bridge" between an inherently meaningless reinforcer and gives it meaning through association. However, since you're likely not being consistent and intentionally pairing the real reward with your pat or verbal reward, it probably won't become very strongly associated with any sort of reward that the horse wants to work for.

However, just for kicks and giggles, let's assume that horses find pats and being told "good boy" or "good girl" is very rewarding for a horse.... It would still be a terrible reinforcer, much in the same way simply "treat training" is a terrible reinforcer and for the same reasons. The trouble with treat training is that you cannot give the horse immediate feedback on specific behaviors since it's impossible to give them a treat at that moment. Usually, if it's impossible to to feed a treat, it would probably be impossible to give them a pat. Thus, it's not really connected to the specific behavior you're working on but rather an overall "I did something right."

The ultimate test to find out whether your pats or words are real reinforcers would be to stop giving them and keep everything else you're doing exactly the same except. If you stopped patting or saying "good boy", would your horse still work for you at the same level/speed he does now? My bet would be yes - because he's not working for the pat or words. Rather, he's working for the release of pressure, the real reinforcer. Thus, since the pats aren't actually motivating the horse to perform the desired behaviors more often, it, by definition, is not a reinforcer at all.

(Disclaimer - I'm not saying you shouldn't pat/pet/rub your horse or tell them "good boy". In fact, though I don't pat because that just isn't something I do for whatever reason, I do give lots of rubs and verbal "good"-s because I do think it reinforces my relationship with my horse. I don't expect it to assist with my training beyond simply establishing a bond with my horse and being comfortable and happy around each other. In contrast, I expect the reinforcement with the clicker to actually produce results in our training.)
PunksTank likes this.
     
    02-14-2013, 12:15 AM
  #17
Yearling
Quote:
Originally Posted by Shropshirerosie    
Subbing, because I know that I will want to use some aspects of clicker training this spring, and this seems like a great resource to come to.
Quote:
Originally Posted by G8tdh0rse    
I have been wanting to try clicker training with my long yearlings. Just to see what it can do. They are curious friendly things now and seem interested in people. I have a clicker book but I am not sure if it is Kirklands.
Quote:
Originally Posted by PunksTank    
JillyBean - I'm loving reading everything so far! It sounds to me like me and you do everything very similar!!
There's only a couple things I do differently - If you don't mind I'll just mention them so everyone reading can see some options in case one way doesn't work too well for them. :)

I'm learning so much in your threads - often while working with them I know that it's working, but I don't know why. So thank you for explaining!!
Quote:
Originally Posted by tiffanyodonnell    
This is very good JB!
Quote:
Originally Posted by Captain Evil    
I am greatly enjoying reading these posts... It must be a lot of work, but I am very interested in the process, progress, and imput!
Hehehe - you all have positively reinforced me on an variable ratio schedule (since I never know when someone is going to say something!), thus contributing to my motivation for continuing this thread. Thank you! :) <3
     
    02-14-2013, 12:26 AM
  #18
Started
JillyBean - I'm very impressed - I'm loving reading about why and how all of the things I do work xD I know how to do it, but not how or why it works.
Thank you so much! This is clearing up so many questions in my mind.

I just wanted to add - to be clear to anyone reading this thread, with all the statements about reinforcement schedules - this Still Applies with pressure+release training without a clicker. Even if all you use is pressure+release, the schedules still take place. This is why when someone has heavy hands with slow or no release of pressure (holding heavy contact all the time) horse's develop 'hard mouths'. The horse's mouth isn't getting tougher and you don't need a stronger bit - you need softer hands and better timed releases of pressure, where the release is significant enough to make the horse desire it.

So the concepts of schedules work for every reinforcer. I just didn't want someone to read this thread and think "well why bother teach CT if you just have to wean them off it" - well (this is my bratty opinion) why bother use something that you need to reinforce All the time? ;)
Don't get me wrong, personally I use a mix of all 4 - Just making a point. :)
jillybean19 likes this.
     
    02-14-2013, 12:29 AM
  #19
Yearling
Backing up, day 2

OK, so now that I've finally gotten all that theory and technicality stuff down, I can finally update on our progress today!

I went out with the goal to just work on what we started yesterday (backing up with a verbal cue), adding speed and getting him to respond to the verbal cue. AND, per PunksTank's suggestion, I wanted to make sure I kept our training session short.

I was pleasantly surprised with how yesterday's lesson apparently "sunk in" overnight! I'm betting that the same thing would have happened even with just a short break yesterday like PunksTanks suggested. Unfortunately, I board my horses so it's a little difficult to spread out our sessions with breaks, but I'll have to get creative. For now, I'll just do little mini-lessons. I'm not sure how long I was out there today, but I made a point of stopping while we were ahead and keeping it shorter than yesterday.

I turned Flash out in the arena as soon as we got out there. He was eager to find out what game we were playing today, so he followed me wherever I went and stopped respectfully when I did (we've worked on where he's supposed to walk respectfully before and he got a reminder the other day when I reacted by shaking his halter without the clicker - he's been very respectful since). Then, I turned around and said "back up" - and he took a step backward! I immediately clicked and treated. He's backing up about 50% of the time on just the verbal cue now and will continue backing up if I keep saying it (backupbackupbackup...). He'll even do so at a decent speed, through I still want to get him faster. If I pick up my energy and walk toward him, shaking my finger at his chest like I did yesterday, then he picks up speed and moves pretty well.

I forgot to mention yesterday how he was swinging his hip some and not backing up straight, but I fixed that by swinging the lead rope at his hip and turning his head slightly, so he straightened back out. He seems to have worked the "straight" thing out now, especially since we're picking up speed and he has to move fairly straight in order to do so quickly.

After a few minutes of backing, he wandered off. I think he's feeding off some other cue he's not quite understanding and that I'm not trying to give, because he basically lunged himself on his own for a while. That alerted me to the fact that I needed to teach him a "come here" cue since he was so convinced he was supposed to be going around me (I try to do most of our training at liberty and didn't have the lead on to stop him). So, for the next few minutes, I focused on just asking him to come. Essentially, I called his name and extended the back of my hand to him and had him target it. Pretty soon, I could send him off by swinging the lead rope and then ask him to come in and touch my hand. Once I had his attention again, I asked him to back up a few steps, then come back forward when I called him and extended my hand. We only did this a few times, and then I decided it was a good place to stop while he was still interested and paying attention.

Tomorrow, I think I'll continue working on the "back up" and "come here cues" and focus on those until we have them really well :)

One last interesting note - we worked a LOT on leading last year out of necessity, including trotting when asked. He knows his cue very well, even after he had the winter off, and immediately trotted up to me when I asked him to catch up while leading him to the arena. However, he never passed me and slowed down as soon as his head was at my shoulder. It's so nice to have a cutie trotting after me and managing the slack in the lead rope appropriately!
     
    02-14-2013, 12:33 AM
  #20
Started
That's fantastic Jillybean! I'm so glad to hear it - yup it sunk in!
Personally I do about 10 minutes of CT, then some barn chores, then a few more minutes. When I work with my mini-pair I'll work with 1 for about 10 minutes, then the other, then back to the first - usually taking an extra few minutes to reload my pockets :) Usually that's enough time for them to have progressed a bit.

Also - I really want to teach my horses that, I'm having some trouble teaching lunging, as it takes a while to click and treat. I have to use a line all the time because my 'arena' has no fence, but it's the only area without grass xD But there's a large hay field right behind it - so I'm sure without the line my horses would be half way across the field before I realized they were gone xD
     

Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Set of Horse training Books- clicker training! jklfarm Tack and Equipment Classifieds 1 12-22-2012 05:13 PM
What do YOU think of clicker training Bribrigirl111 Horse Training 93 08-31-2012 03:18 AM
Clicker training Semiellia Horse Training 22 06-06-2012 02:55 AM
clicker training? MarleyandEllie Horse Training 18 11-29-2010 04:02 AM
Clicker Training??? BluMagic Horse Training 4 01-21-2008 02:07 PM



All times are GMT -4. The time now is 02:25 AM.


Powered by vBulletin® Version 3.8.5
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Search Engine Friendly URLs by vBSEO 3.6.0