# Primary Reinfrocing Secondaries



## GoldenSail (Dec 30, 2008)

For anyone interested here is a few of the articles I found doing a rough search. 

My biggest critic is that I could not find what I really wanted. It seems the secondaries they are using are abstract things (such as light, a click, etc) associated with food, not something that stands on its own as being a reinforcer. I do not believe praise and play need to be associated with a primary for them to be reinforcing for a dog, so seeing this data does not convince me  For those of you really into the science--why does it seem that praise/play are considered secondaries as is a conditioned clicker associated with a primary? They are sooo different!

If anyone can find it, this would be my ideal study. Using dogs, you would have a group getting reinforcement with secondaries and those secondaries backed by primaries, and a group that only received secondaries, and a group that received secondaries and primaries but the two reinforcement types were not associated with each other. All the studies I saw did not use a group based solely on secondaries, they all had some correlation to a primary.



The relative effectiveness of secondary reinforcers throughout deprivation and habit-strength parameters.
Journal of Comparative and Physiological Psychology, Vol 49(2), Apr, 1956. pp. 126-130
Miles, Raymond C., Ohio State U., Columbus.
All animals extinguished with the secondary reinforcers present took longer to extinguish than did animals extinguished without the secondary cues. Rats given uniform reinforcement during training and extinguished under varying degrees of deprivation, with and without secondary reinforcers, also showed that extinction in the presence of secondary reinforcement took longer through the whole deprivation range. "It is concluded that the proportion of extinction responses attributed to the secondary reinforcers was relatively constant throughout both the habit-strength and deprivation parameters." (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Fantino, E (01/1968). "Secondary reinforcement and number of primary reinforcements.". Journal of the experimental analysis of behavior (0022-5002), 11 (1), p. 9.
“Hernstein (1964) showed that the strength of a secondary reinforce, as measured by its effectiveness in a chained schedule, depended more heavily on the rate than on the probability of primary reinforcement in its presence”
“Still another indication that number of reinforcements does not affect the strength of a
secondary reinforcer in the same manner as rate of reinforcement,”
The efficacy of a secondary reinforcer (clicker) during acquisition and extinction of an operant task in horses.
Williams, J. L.; Friend, T. H.; Nevill, C. H.; Archer, G.; Applied Animal Behaviour Science, Vol 88(3-4), Oct, 2004. pp. 331-341. [Journal Article] Abstract: "Clicker training" is a popularly promoted training method based on operant conditioning with the use of a secondaryreinforcer (the clicker). While this method draws from theories of learning and is used widely, there has been little scientific investigation of its efficacy. We used 60 horses, Equus callabus, and assigned each horse to one of six reinforcement protocols. The reinforcement protocols involved combinations of reinforcers administered (primary versus secondary plus primary), schedule of reinforcement (continuous versus variable ratio), and reinforcers applied during extinction (none or secondary). There were no differences (P≥0.11) between horses which received a secondary reinforcer (click) followed by the primary reinforcer (food) and those which received only the primary reinforcer (food) in the number of trials required to train the horses to touch their noses to a plastic cone (operant response). There also were no differences (P≥0.12) between horses which received the secondary reinforcer plus primary reinforcerand those which received only the primary reinforcer in regards to the number of trials to extinction. We conclude that there is no difference in the amount of training required to learn the operant task or in the task's resistance to extinction between horses that received a secondary reinforcer followed by a primary reinforcer versus horses which received only a primary reinforcer. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Secondary reinforcement as affected by reward schedule and the testing situation.
D'amato, M. R.; Lachman, Roy; Kivy, Peter; Journal of Comparative and Physiological Psychology, Vol 51(6), Dec, 1958. pp. 737-741.[Journal Article] Abstract: Relatively little is known about the effect of a random partial-reinforcement schedule on the strength of asecondary reinforcer. Aside from an early paper by Saltzman (8) and a recent report (5), there is virtually no other published work directed specifically to this problem. In view of the obvious importance of this variable, it would seem desirable to obtain additional relevant data. A second, concurrent purpose is to determine whether the disparity between "single-group" and "separate-groups" designs noted earlier (2) extends to this variable. Previous work (4) had indicated that amount of primary reward does not significantly affect the strength of a secondary reinforcer when separate groups of Ss are used for the high and low reward conditions. However, it was later reported (2) that if the same S is made to choose between two cues formerly associated with different amounts of primary reward (single-group design), a distinct preference develops for the stimulus previously correlated with the relatively large reward, indicating that under the latter conditions amount of reward is a significant variable. Both types of designs are utilized in the present report. Experiment I: In this experiment the separate-groups design was employed. One group of Ss received 100% reinforcement in a characteristic end box of a straight alley, while a second group was rewarded on only 50% of the trials, randomly. It has been suggested that the former schedule should result in a secondary reinforcer with greater "instrumental value" (7), or one leading to a more rapid rate of learning of a new response (6), but one, nevertheless, which succumbs to extinction more rapidly than a secondary reinforcer established under partial reinforcement. Sufficient testing trials have been included in the present experiment to produce data bearing on this question. Experiment II: In this experiment, a single-group design, each animal serves as its own control. For each S one secondary reinforcer is established under continuous reward training, while a second is established under partial reinforcement; during testing, S is required to choose between the two. Reasoning from the results of Experiment I, we might expect that for the first several testing trials (or even for the first testing day) the choices would be rather evenly divided, with a preference subsequently emerging for the secondary reinforcerdeveloped under the partial-reinforcement schedule. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Clicker training in horses: Operant conditioners and secondary reinforcers in horse training. Williams, Jennifer Lee; Dissertation Abstracts International: Section B: The Sciences and Engineering, Vol 63(8-B), Mar, 2003. pp. 3521.[Dissertation] Abstract: "Clicker training", a training method based on operant conditioning with the use of a secondary reinforcer, has gained popularity as horse owners and trainers seek improved ways of training their horses. While this method draws from theories of the psychology of learning and is widely used by many owners, there has been no scientific investigation of its efficacy. This study was conducted to assess the hypothesis that operant conditioning with a secondary reinforcer (auditory click) followed by a primaryreinforcer (food) results in fewer target training trials to learn an operant task and more trials to extinction than operant conditioning with only a primary reinforcer. Sixty horses were divided into one of six treatments based on which reinforcers were administered during conditioning, target training, and extinction. The operant response required the horse to touch his or her nose to a plastic cone. The number of shaping trials required, the total number of target training trials required, and the total number of trials required to extinction were not influenced by type of reinforcer(s) administered (P ? 0.33), age (P ? 0.08), or sex (P ? 0.14). The total number of target training trials and the total number of trials required to extinction were not influenced by schedule of reinforcement (P ? 0.33). The number of extinction trials required was not influenced by the reinforcers withheld during extinction (P = 0.38). This study failed to find a difference between horses which received a secondary reinforcer followed by a primary reinforcer and those which received only a primary reinforcer in regards to the amount of time required to train a response or in the number of trials required to reach extinction. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Secondary reinforcement effects of combining food with praise in dogs. Travers, Mary E.; Dissertation Abstracts International: Section B: The Sciences and Engineering, Vol 70(10-B), 2010. pp. 6590.[Dissertation] Abstract: Many researchers use food as a positive reinforcer for specific behavior change. Although food is necessary for survival, there are instances where food intake in both animals and humans can be excessive. It is plausible that overeating is the result of learning to associate food with other reinforcers. The purpose of this study was to investigate whether food would function as a conditioned (i.e. secondary) reinforcer in dogs. Subjects were 40 adult small breed (7-251bs) dogs between the ages of one and eight. The study was a 4 x 2 random assignment design consisting of five phases (ABCA'D). Dogs were randomly assigned to one of four reinforcement groups; food and praise, food only, praise only and a non-contingent control group. The mean difference in amount of food consumed from baseline one to return to baseline showed significant differences (p = 0.038). Group mean differences in average weight change from baseline one to return to baseline also reached statistical significance (p < 0.01). Dogs reinforced with food plus praise were significantly heavier than dogs reinforced with food alone, praise alone or the non-reinforcement control. Dogs reinforced with food plus praise demonstrated fewer trials to criterion and a larger number of responses to food refusal, as hypothesized; however these differences failed to reach statistical significance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Secondary reinforcement and magnitude of primary reinforcement. D'Amato, Michael R.; Journal of Comparative and Physiological Psychology, Vol 48(5), Oct, 1955. pp. 378-380. [Journal Article] Abstract: Rats were given 70 trials in a straight alleyway under 23 hour food deprivation; 35 trials were rewarded with high reward (5 food pellets) and 35 with low reward (1 pellet) in discriminably different goal boxes. When non-rewarded test trials were run on a T maze with the high reward box on one side and the low reward on the other, it was found that the mean number of responses to the high reward side was significantly greater. "It was concluded that, where one secondary reinforcer is pitted against another, the amount of primary reward with which each derived reinforcer was previously correlated is an important variable determining the relative strength of the secondaryreinforcers." (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Relation of amount of primary reinforcement to discrimination and to secondary reinforcement strength. Stebbins, William C.; Journal of Comparative and Physiological Psychology, Vol 52(6), Dec, 1959. pp. 721-726. [Journal Article] Abstract: Thirty-six male rats of Wistar strain were divided at random into six equal groups and maintained on 22 hr. of food deprivation throughout the experiment. Each group was given light-dark discrimination training on the bar-press response for one of six sucrose solutions ranging from a 5% concentration to a 50% concentration. Reinforcement was administered on a 1-min. variable-interval schedule in the light, and reinforcement was withheld altogether in the dark. Following discrimination training, all animals were given further extinction in the dark until all had reached the same level of responding, At this point the light was reintroduced for 1 sec. following each response. Following this procedure, all animals were extinguished in the presence of the light until all had reached the same level of responding. The sound of the liquid feeder (dipper) was then reintroduced following each response. The rate-concentration function in the light was ostensibly linear between the 5% and the 32% sucrose concentration but dropped sharply between the 32% and the 50% concentration. By nonparametric one-way analysis of variance the function was significant (p < .005). The response rate in the dark was well below the rate in the light and was apparently unaffected by changes in the value of the reinforcing agent, rate differences between concentrations being statistically unreliable (p > .90). The rate-concentration function for the light as a secondary reinforcer appeared to be monotonic and negatively accelerated and to approach an asymptote between the 32 % and the 50% concentration. By nonparametric analysis of variance the function was significant (p < .01). The test for the dipper sound as a secondary reinforcer yielded a relationship to the sucrose concentration similar in shape to the function obtained with the light. Application of the same statistical test indicated significant over-all differences


----------



## Loisiana (Jul 29, 2009)

I will admit I am not all that into the science part of training. But for kicks I just looked up the definitions of primary and secondary reinforcers to see if I was missing something. Now this was is only based on about five minutes of research, but I found it interesting that on the sites I looked at that were aimed specifically on dog training, they defined primary reinforcers as those things needed to survive, but on websites that were defining those terms in a more general sense (not specific to dog training), they defined primary reinforcers as those reinforcers that did not require pairing with another reinforcer.

So I figure there's three reasons I'm possibly seeing a difference here: a)I didn't look at enough sites, and it just so happens that those definitions were on those sites but if I would have looked longer I would have gotten different result averages or b) animal trainers have taken the liberty of using their own definitions or c) it is assumed that if it isn't necessary for survival then it has to be taught to be reinforcing.

I'm thinking about the use of praise. I often say "yes" in a neutral tone to provide information that the dog is correct. That is obviously a secondary reinforcer, I had to teach the dog that "yes" was a good thing. But I can go to a baby puppy and talk to it in a squeaky happy voice and the puppy will wriggle around in joy. So wouldn't that be a primary reinforcer? It doesn't need that squeaky voice to survive, but the voice doesn't have to be paired with anything else to have taught it to like that sound.

Have any of you seen the test where they tickled rats at the end of a maze? Those rats that were tickled when they completed the maze would finish the maze much quicker on future repetitions than rats who were not tickled. That would lead one to believe that the rats found the tickling to be reinforcing, yet the tickling had not been previously paired with anything so wouldn't that make it a primary reinforcer?


----------



## GoldenSail (Dec 30, 2008)

Yes, what is classified as primary or secondary is very confusing. All of the studies I found where the primary strengthened the secondary involved food as the primary and the secondary was a conditioned secondary--something that apparently had no value before the association.

Funny you mention the rats--I was just thinking about that! It would be a good way to make up a study of what I am looking for. Use tickling as a secondary, have a group that is only ticked, and one that is tickled and given food simultaneously, and one that is given food and tickling but not at the same time. Then over time observe the behavior of the rats and reliability. Do rats that previously got food with their tickling respond differently when the food is put on a variable schedule? Will they work as long when food is removed and only get tickled?


----------



## tanianault (Dec 11, 2010)

According to the clickertraining glossary on the Karen Pryor website: Clicker Training Terms | Karen Pryor Clickertraining

"Primary reinforcer: A reinforcer that the animal is born needing. Food, water, and sex are primary reinforcers." 

"Secondary reinforcer: A conditioned reinforcer. A reinforcer the animal is not born needing. Secondary reinforcers may be as, or even more, powerful than a primary reinforcer." 

"Conditioned reinforcer: A neutral stimulus paired with a primary reinforcer until the neutral stimulus takes on the reinforcing properties of the primary. A clicker, after being repeatedly associated with a food treat or other reinforcer, becomes a conditioned reinforcer."

My guess as to why praise and play are considered secondary reinforcers is because they are not technically something the animal requires to survive, at least not in the same way as food and water are required to survive. (Although, I do seem to remember a number of studies with monkeys that demonstrated that play is required for the animals to thrive.)

- Tania


----------



## GoldenSail (Dec 30, 2008)

tanianault said:


> According to the clickertraining glossary on the Karen Pryor website: Clicker Training Terms | Karen Pryor Clickertraining
> 
> "Primary reinforcer: A reinforcer that the animal is born needing. Food, water, and sex are primary reinforcers."
> 
> ...


Yes--but who made her the authority?  That is generally how I have seen things, but the scientific papers I have been reading seem to equate a conditioned reinforcer as a secondary reinforcer, but I see distinct differences between them.


----------



## Loisiana (Jul 29, 2009)

I have another question.....why is sex so often listed as something necessary for survival? It obviously isn't necessary for the survival of the individual (although some males might disagree ). Or is it referring to the survival of the species as a whole? It doesn't make sense to me....if you define a primary reinforcer as something needed for survival and give sex as an example of a primary reinforcer, how does that fit together when an individual can live their entire lives without it? But if you define primary reinforcer as something that is instinctually enjoyed without having to be taught then it makes sense to me.

I think Flip is at the age where he would really, really like to volunteer if anyone wants to run any experiments :uhoh:


----------



## Megora (Jun 7, 2010)

tanianault said:


> "Primary reinforcer: A reinforcer that the animal is born needing. Food, water, and sex are primary reinforcers."
> 
> "Secondary reinforcer: A conditioned reinforcer. A reinforcer the animal is not born needing. Secondary reinforcers may be as, or even more, powerful than a primary reinforcer."
> 
> "Conditioned reinforcer: A neutral stimulus paired with a primary reinforcer until the neutral stimulus takes on the reinforcing properties of the primary. A clicker, after being repeatedly associated with a food treat or other reinforcer, becomes a conditioned reinforcer."


I like how simply and basic these are.  

Then again, I'm thinking you could have a full discussion as to where the following get categorized, depending on the trainer:

Food Rewards
Praise Rewards
Play/Attention Rewards
Cues (clickers, hand signals, body language, etc)

Leash or Collar Corrections
Verbal Corrections
Physical Corrections
Reinforced Hierarchy

^ I was thinking that some trainers could and would read into the natural needs of an animal being food, water, and sex and assume that only means GOOD and fluffy things. 

But animals of all kinds require a certain hierarchy and rule establishing that they enforce or comply with. And it is a life and death issue for them. So that goes back to the old way of training using methods that mimick the natural reinforcers that are the very first things a puppy learns from it's littermates and mother. 

So (to me) food and correction may be primary reinforcers. Praise and play are secondary. And conditioned reinforcement might be my body language and other visual cues. <- But honestly speaking, when it comes to having a happy dog who excels at something as unnatural as competition obedience, I would roll all of those things together.

@Jodie - My immediate thought is that it can't be a strong reinforcer for... well, for a neutered dog.  Then I did remember something as it applies to birds. If you want to tame a bird and teach it tricks, you have to make it think that you are a bird and even its mate. This means isolating that bird from the other birds you might have or keeping it alone so that it's singing and bonding with you instead of another bird. 

I'm not sure how that applies to dogs other than thinking that some people may report a stronger bond with dogs that were raised in single dog households? Possibly...?


----------



## Loisiana (Jul 29, 2009)

who was it that had the video showing her allowing her dog to hump her leg as its reinforcer? yeah, that one won't be happening here...


----------



## Megora (Jun 7, 2010)

Loisiana said:


> who was it that had the video showing her allowing her dog to hump her leg as its reinforcer? yeah, that one won't be happening here...


:yuck:

I think a video like that would make me want to bleach my eyes and my legs.


----------



## GoldenSail (Dec 30, 2008)

Megora said:


> So (to me) food and correction may be primary reinforcers. Praise and play are secondary. And conditioned reinforcement might be my body language and other visual cues. <- But honestly speaking, when it comes to having a happy dog who excels at something as unnatural as competition obedience, I would roll all of those things together.


Corrections are not reinforcers of anything--reinforcers increase the frequency of the behavior and punishers decrease the behavior. When you give a correction it is because your dog did something wrong and you are trying to decrease that wrong response (punisher) nor increase it (reinforce)


----------



## Megora (Jun 7, 2010)

GoldenSail said:


> Corrections are not reinforcers of anything--reinforcers increase the frequency of the behavior and punishers decrease the behavior. When you give a correction it is because your dog did something wrong and you are trying to decrease that wrong response (punisher) nor increase it (reinforce)


But are corrections (leash pops, for example) always punishment? 

I mean, the way I taught Jacks the swing was using what other people might consider a correction. I used both the leash and a toe nudge to get him to jump into position. 

Same thing with how I taught "get it in" and left turns. You put the leash behind you and tug it the same time you give the verbal command and you only treat when the dog does what you want.


----------



## Chelseanr (Oct 3, 2010)

Am I understanding this correctly that secondary reinforcers such as a clicker have no effect on how quickly an animal will pick up a behavior?


----------



## GoldenSail (Dec 30, 2008)

And if anyone is wondering why this thread exists, it's mostly my interest at the idea of associating food (primary) with praise (secondary) to strengthen the value of praise. Personally, I am not convinced that food matched with praise makes praise a stronger reinforcer and suspect that it could make it less strong...


----------



## The_Artful_Dodger (Mar 26, 2009)

In my psychology classes we learned that a reinforcer can only be defined by the effect that it has on the subject - something that makes the target behaviour more likely to occur is a reinforcer. 

A primary reinforcer is something that provides reinforcement without needing to be learned - food is naturally reinforcing because there is a natural drive to eat when hungry. You don't need to teach a hungry animal that food is a good thing, or a thirsty animal that water is a good thing. 

A secondary or conditioned reinforcer is a stimulus that isn't naturally reinforcing, but that takes on the qualities of the primary reinforcer after consistent pairings. (e.g., clicker training).

I can see praise as being a primary reinforcer and as a secondary reinforcer. One could argue that animals have a need for social interaction so praise/attention could be naturally reinforcing. It's just not as obvious or easy to manipulate as food and water 

Praise could also be considered a secondary reinforcer if a primary reinforcer (e.g. food reward) is consistently paired with priase. In this way the animal has been taught that praise is associated with getting food, and then the praise itself becomes reinforcing. 

As for sex as a primary reinforcer, I don't think it has to do with it being necessary for survival of the individual animal - but because it is necessary for survival of the species there is a natural drive for sex - you don't need to teach the animal to enjoy sex.


----------



## tippykayak (Oct 27, 2008)

GoldenSail said:


> And if anyone is wondering why this thread exists, it's mostly my interest at the idea of associating food (primary) with praise (secondary) to strengthen the value of praise. Personally, I am not convinced that food matched with praise makes praise a stronger reinforcer and suspect that it could make it less strong...


I don't really care about the semantic part of the argument, but I think food can go both ways. You can definitely misuse it, and then your dog starts ignoring praise because it's getting in the way of the cookie. Or, if it's truly sporadic and especially if it's produced out of nowhere at the right time, it can make the game more fun, since the game produces all of these exciting, happy reactions, of which food is just one.


----------



## GoldenSail (Dec 30, 2008)

tippykayak said:


> I don't really care about the semantic part of the argument, but I think food can go both ways. You can definitely misuse it, and then your dog starts ignoring praise because it's getting in the way of the cookie. Or, if it's truly sporadic and especially if it's produced out of nowhere at the right time, it can make the game more fun, since the game produces all of these exciting, happy reactions, of which food is just one.


Not sure what you mean by the semantic part. Anyway, even if used on a variable schedule are you not then at risk for losing the association (Primary-->secondary) that you wanted anyway?


----------



## tippykayak (Oct 27, 2008)

GoldenSail said:


> Not sure what you mean by the semantic part. Anyway, even if used on a variable schedule are you not then at risk for losing the association (Primary-->secondary) that you wanted anyway?


By semantics, I mean what we call them. Are they secondary because they're neutral (click) or because they're not related to survival (praise)? I'm not so interested in figuring out what to call them.

And variable (sporadic) schedules are actually more enforcing of associations than predictable schedules. So the dog actually associates _more_ pleasure and reward with a game when the rewards have been sporadic. I go by the rule of always praise but use food, toys, and games sporadically, especially once a behavior is already shaped.


----------



## GoldenSail (Dec 30, 2008)

tippykayak said:


> By semantics, I mean what we call them. Are they secondary because they're neutral (click) or because they're not related to survival (praise)? I'm not so interested in figuring out what to call them.
> 
> And variable (sporadic) schedules are actually more enforcing of associations than predictable schedules. So the dog actually associates _more_ pleasure and reward with a game when the rewards have been sporadic. I go by the rule of always praise but use food, toys, and games sporadically, especially once a behavior is already shaped.


Gotcha--I do think it is important to differentiate the two especially when using scientific literature to back up ideas--such as secondaries become stronger when associated with primaries. All the literature I found involved using secondaries that had no inherent value without the primary whereas praise in many cases does. So if I take secondary to mean praise/play, primary to mean food/sex and then try to use scientific literature as the basis for my argument which uses a different definition--it doesn't work!

And yes I do know variable is stronger--but I don't think I've seen anyone condition a clicker only to later use it on a variable schedule. I've heard of it, not sure how successful it is--but with a conditioned reinforcer like a click I've always heard click=treat even if you make a mistake. So back to using a primary to reinforce a secondary--if say you are using food with praise. If you are only using food sometimes and praise always, then don't you think you are going to lose the association that link praise and food together that supposedly strengthens praise alone? And if you are not on a variable schedule, the dog will likely be disappointed if s/he receives praise but no food when in the past praise has always been associated with food which is supposedly a stronger reinforcer.


----------



## The_Artful_Dodger (Mar 26, 2009)

GoldenSail said:


> Not sure what you mean by the semantic part. Anyway, even if used on a variable schedule are you not then at risk for losing the association (Primary-->secondary) that you wanted anyway?


Using a variable schedule does not mean that you stop presenting the primary reinforcer. When I trained my lab rat (Patches), we used food pellets as a primary reinforcer and used a clicking sound as a secondary reinforcer. We experimented with various schedules of reinforement - changing how often she would be rewarded for performing the target behaviour (e.g., pressing a bar) - but everytime we clicked she got a food pellet. And everytime I click Dodger's clicker he gets a treat - but that doesn't mean I click everytime he does the desired behaviour (unless it is something brand new I am training). 

In repsonse to your question about food rewards dimishing the effect of praise, I can see it both ways, so I guess it depends on what you mean. 

On the one hand, if you are using a cue word as praise (Yes or Good or whatever), then pairing it with food works the same a clicker training. 

On the other hand, if you are saying that praise is intrinsically motivating for the dog, then I can see how using food might make it less effective. If food is more motivating to the dog than the praise then the dog may pay less attention to the praise because they are focused on the food.


----------



## tippykayak (Oct 27, 2008)

GoldenSail said:


> Gotcha--I do think it is important to differentiate the two especially when using scientific literature to back up ideas--such as secondaries become stronger when associated with primaries. All the literature I found involved using secondaries that had no inherent value without the primary whereas praise in many cases does. So if I take secondary to mean praise/play, primary to mean food/sex and then try to use scientific literature as the basis for my argument which uses a different definition--it doesn't work!


I agree that the differentiation is important, but the names have gotten really tangled in the dog literature, so I didn't feel like arguing about what each kind ought to be named. 



GoldenSail said:


> And yes I do know variable is stronger--but I don't think I've seen anyone condition a clicker only to later use it on a variable schedule. I've heard of it, not sure how successful it is--but with a conditioned reinforcer like a click I've always heard click=treat even if you make a mistake.


I think the difference is that the clicker is not actually being used as a reinforcer in itself (secondary or otherwise). It marks the reinforcement, letting the dog pair the actual reward (food) more accurately with the behavior caused the food to come. Because of that association, the clicker might become a true reinforcer on its own, but I don't know of anybody who uses it that way.



GoldenSail said:


> So back to using a primary to reinforce a secondary--if say you are using food with praise. If you are only using food sometimes and praise always, then don't you think you are going to lose the association that link praise and food together that supposedly strengthens praise alone? And if you are not on a variable schedule, the dog will likely be disappointed if s/he receives praise but no food when in the past praise has always been associated with food which is supposedly a stronger reinforcer.


Because praise acts as both marker and reinforcer, food, according to my understanding of basic Skinnerian principles, would be paired with praise, and the praise would bring many of the same feelings to the dog that the food does. By pairing them sporadically, you're both teaching the dog to work for praise and strengthening the food-based feelings that you've associated with the praise.

My personal experience backs that up. My dogs seem to be more and more enthusiastic about working for praise itself, and since food is sporadic and produced from nowhere, they don't automatically look for a treat after each successful skill.

I've never had a dog seem "disappointed" to get praise but no treat after a successful execution of a skill.

I should say that my experience is highly biased towards very biddable Golden Retrievers who love to work and are very easy to motivate with excitement, games, and praise. Starting out with dogs, I didn't use food at all, but I've since found that it can be a very helpful piece of the toolkit, so I use it pretty consistently now.


----------



## Loisiana (Jul 29, 2009)

tippykayak said:


> And variable (sporadic) schedules are actually more enforcing of associations than predictable schedules. So the dog actually associates _more_ pleasure and reward with a game when the rewards have been sporadic. I go by the rule of always praise but use food, toys, and games sporadically, especially once a behavior is already shaped.


But isn't that still producing a predictable reward schedule of praise? Not arguing against it, just curious what the thoughts are if it has been proven that variable schedules are more enforcing than predictable schedules why this would not apply to praise.


----------



## The_Artful_Dodger (Mar 26, 2009)

tippykayak said:


> I agree that the differentiation is important, but the names have gotten really tangled in the dog literature, so I didn't feel like arguing about what each kind ought to be named.
> 
> 
> 
> I think the difference is that the clicker is not actually being used as a reinforcer in itself (secondary or otherwise). It marks the reinforcement, letting the dog pair the actual reward (food) more accurately with the behavior caused the food to come. Because of that association, the clicker might become a true reinforcer on its own, but I don't know of anybody who uses it that way.


I don't want to argue semantics either, but I do think it is confusing if everyone is using different definitions. 

Anyways, I think the clicker is being used as a reinforcer (secondary) because (after properly teaching the association between the click and the primary reinforcer) the behaviour that the subject (rat, dog, whatever) is exhibiting at the time of the click will become more frequent. The animal is making an association between the click and the behaviour, and if they have already been conditioned to associate the click with food, then the click does reinforce the behaviour.


----------



## RedDogs (Jan 30, 2010)

Too much to reply to and not enough time... I'll try to do a little now!

- My understanding is that a variable schedule doesn't increase the amount/quantity/quality of the reinforcer, but that it makes the behavior more resistant to extinction.
- That said... I also understand that an -unexpected- reinforcer changes the neuron firing and can increase the quality/quantity of the behavior. But "unexpected" is the key part there.
- As I said in the other post.... toy play with -dogs- may be physiologically associated with food. The idea makes sense, though I don't know of any studies to back it up. But this could arguably make toys a subset/type of/potential primary for some dogs. That said, we also need to consider its value as a stress relief and/or cue that punishment is not likely to happen in some training contexts.
- Classical conditioning is a type of 'learning by association'. There is specific patterns of neuron firing when this is occuring (and when the expected thing does...or does not! happen). We can teach dogs to have a variety of conditioned/secondary reinforcers: specific types of petting, clapping, specific words...etc.... 
- But what does pairing talking with food actually do? It depends. It won't always increase the value of your talking, it could decrease. But the thing is to select words/phrases/inflections/things/associated body language that do NOT indicate other things. We have some things that indicate going for walks, going outside, getting water, going swimming, going to class, playing in the yard, going with friends, etc.... and while if we surprise the dog iwth a different reinfrocer after the words....it could increase the value (like..if we went to the ktichen where we eat, but instead of dog food, I pulled out a giant box of hamburgers), but it could also decrease the value of the talking (if I was getting out everything to go swimming and playing fieldy games....and then fed some dog food. Sure the dog food is a primary reinforcer, sure it's reinforcing sometimes, but in this setting it is NOT....).
- So maybe to summerize that piece, we need to look at the hierarchy of cues, the hierarchy of reinforcers, and the context settings, as well as the physiological response?
- When looking at the studies.... it doesn't appear to be much direclty relevant to praise and food specifically with dogs . With the other animals/species, it probably wouldn't even be an option, given our specific strong history of interaction with domestic dogs. 

So... more specifically to what I do
- We have lots of different touching and different phrases. Some of the phrases are associated with specific things. Like going to swim or going for walks ro going for specific things or starting specific games. Due to some very very very clever friends, I'm now using different markers for toy games v food training. I may use food during toy training, but I'm always very careful and distincat about what I do.
I have talking associated with food and talking associated with toy play. I'm strategic about when I use these and for now I'm sticking with the (followed by reinforcer) 9/10 that Ken R recommended. I want to have ltos of different types of talking and types of touching so that when we're in situations (cough, obed, cough) where I'm limited by what I do....and the rienforcement available, I have a vareity of options available.

Now back to work...........


----------



## tippykayak (Oct 27, 2008)

Loisiana said:


> But isn't that still producing a predictable reward schedule of praise? Not arguing against it, just curious what the thoughts are if it has been proven that variable schedules are more enforcing than predictable schedules why this would not apply to praise.


Exactly, yes. Because praise is also the marker, I use it every time. It would not produce as strong an association (as RedDogs clarified, more extinction-resistant) if that's all I did. By being sporadic with other things, I try to get the best of both worlds: the necessary consistency of a marker and the extra strength of a variable reinforcement schedule.


----------



## tippykayak (Oct 27, 2008)

The_Artful_Dodger said:


> I don't want to argue semantics either, but I do think it is confusing if everyone is using different definitions.
> 
> Anyways, I think the clicker is being used as a reinforcer (secondary) because (after properly teaching the association between the click and the primary reinforcer) the behaviour that the subject (rat, dog, whatever) is exhibiting at the time of the click will become more frequent. The animal is making an association between the click and the behaviour, and if they have already been conditioned to associate the click with food, then the click does reinforce the behaviour.


It's true, and it does, but I don't think that's the central purpose of the clicker. People don't use it to take over the need for a primary reinforcer. Its main use is as a marker, not as a reinforcer in itself (though it surely does function that way, or why else would you "load" it?). I bet you could use it as the only reinforcer, but that wouldn't be mainstream clicker training.


----------



## FlyingQuizini (Oct 24, 2006)

Megora said:


> But are corrections (leash pops, for example) always punishment?
> 
> I mean, the way I taught Jacks the swing was using what other people might consider a correction. I used both the leash and a toe nudge to get him to jump into position.


Your example with the leash and toe nudge would be negative reinforcement. You added something the dog worked to avoid. As soon as the dog did what you wanted, you relieved the pressure of the leash and your toe.

If the goal of a leash pop is to decrease something (dog jumps on visitor, handler yanks on leash), it's a punisher. ONLY IF, of course, it actually decreases the unwanted behavior. If it has no overall longterm effect (which often happens with positive punishment attempts such as the one I'm describing) then it's really just nagging and not being very nice to the dog!


----------



## The_Artful_Dodger (Mar 26, 2009)

tippykayak said:


> It's true, and it does, but I don't think that's the central purpose of the clicker. People don't use it to take over the need for a primary reinforcer. Its main use is as a marker, not as a reinforcer in itself (though it surely does function that way, or why else would you "load" it?). I bet you could use it as the only reinforcer, but that wouldn't be mainstream clicker training.


 
Oh yeah - I'm not saying to use the clicker on it's own without a primary reinforcer. I'm saying that you teach them first the association between the clicker and food - the clicker becomes a discriminatory stimulus (or a marker) to signal that food will be available. Then when you start using the click to train new behaviours, you are technically using the click as a secondary reinforcer (by operant conditioning definitions anyways).


----------



## tippykayak (Oct 27, 2008)

The_Artful_Dodger said:


> Oh yeah - I'm not saying to use the clicker on it's own without a primary reinforcer. I'm saying that you teach them first the association between the clicker and food - the clicker becomes a discriminatory stimulus (or a marker) to signal that food will be available. Then when you start using the click to train new behaviours, you are technically using the click as a secondary reinforcer (by operant conditioning definitions anyways).


Right, except that you never use it by itself as the lone reinforcer. So it's sort of a hybrid of a marker and a reinforcer, but you're relying on the food to really keep the reinforcement (of the behavior) going.


----------



## The_Artful_Dodger (Mar 26, 2009)

tippykayak said:


> Right, except that you never use it by itself as the lone reinforcer. So it's sort of a hybrid of a marker and a reinforcer, but you're relying on the food to really keep the reinforcement (of the behavior) going.


True. But it doesn't have to be used on its own to be a reinforcer. 

In my psych class "experient" we did five phases;
1 - taught the rat to associate the click with food
2. taught the rat to press a bar to elicit a click and get food
3. taught the rat that bar pressing only elicits a click when the light is on 
4. reinforment schedule changed - 5 bar presses requried (with less than 5 seconds between presses) to elicit a click
5. taught the rat to touch her nose to a trapeze to turn on a light (which would then signal that she could press the bar 5 times to get food)

In this case both the click (which reinforced the bar pressing behaviour) and the light turning on (which reinforced the trapeze touching behaviour) were both considered secondary reinforcers 

(By the way, I'm not trying to be difficult - I just find it interesting)


----------



## GoldenSail (Dec 30, 2008)

The_Artful_Dodger said:


> I don't want to argue semantics either, but I do think it is confusing if everyone is using different definitions.
> 
> Anyways, I think the clicker is being used as a reinforcer (secondary) because (after properly teaching the association between the click and the primary reinforcer) the behaviour that the subject (rat, dog, whatever) is exhibiting at the time of the click will become more frequent. The animal is making an association between the click and the behaviour, and if they have already been conditioned to associate the click with food, then the click does reinforce the behaviour.


Agree about the semantics--and honestly I don't care one way or the other I just want to be consistent.

Now, I think you are misunderstanding me. Yes, a variable schedule means you still use the primary just not every time. However, you don't use a conditioned reinforcer without the primary--do you? Do you click and expect that to be a reward without following up with something else?


----------



## GoldenSail (Dec 30, 2008)

tippykayak said:


> It's true, and it does, but I don't think that's the central purpose of the clicker. People don't use it to take over the need for a primary reinforcer. Its main use is as a marker, not as a reinforcer in itself (though it surely does function that way, or why else would you "load" it?). I bet you could use it as the only reinforcer, but that wouldn't be mainstream clicker training.


Yes, and this is just what I was thinking. All cases I found that supported primary paired with secondary reinforcement strengthened it--those secondary reinforcements were conditioned and by themselves did not seem to hold any value to the trainee. So, therefore I haven't found anything that supports or refutes the idea that pairing praise with food makes praise better. I personally think it could be a slippery slope, and am doubtful that you could take a dog that dislikes praise/petting and get it to work for those things by pairing it with food. 

As far as 'disappointment' maybe I am being lenient, but I have worked with dogs that when you reached out to pet them jerked away because you did not have a cookie for the reward like they expected. This bothers me greatly.


----------



## The_Artful_Dodger (Mar 26, 2009)

GoldenSail said:


> Agree about the semantics--and honestly I don't care one way or the other I just want to be consistent.
> 
> Now, I think you are misunderstanding me. Yes, a variable schedule means you still use the primary just not every time. However, you don't use a conditioned reinforcer without the primary--do you? Do you click and expect that to be a reward without following up with something else?


No. You click based on the schedule of reinforcement that you want to use and it is always paired with the primary reinforcer. I guess my understanding of a secondary reinforcer is that its not meant to replace the primary reinforcer...but its just easier to make the association with the desired behaviour (i.e., get the right timing to mark the behaviour) with a clicker or cue word. The secondary reinforcer makes the behaviour it marks more likely to be repeated...but the drive to do so is still related to the primary reinforcer. 


I wouldn't consider praise on its own to be a secondary reinforcer...because I think of secondary as being a neutral stimulis that is conditioned to be a reinforcer. If the praise is intrinsically rewarding than I would consider it a primary reinforcer. So I would think what you are asking is that by pairing two primary reinforcers (food and praise)- will the weaker primary reinforcer become strengthened by the stronger primary reinforcer? I dont know but I would think no. But the combined effect of the drive for food and drive for praise when used together might increase the overall motivation to perform the desired behaviour.


----------



## Loisiana (Jul 29, 2009)

So what would you call it when you pair something used to change a behavior with a primary reinforcer?


----------



## The_Artful_Dodger (Mar 26, 2009)

Loisiana said:


> So what would you call it when you pair something used to change a behavior with a primary reinforcer?


Do you mean pairing something adversive to decrease a behaviour with a primary reinforcer? I would call that confusing! I would imagine that if you paired say a shock and a food reward as a response to behaviour, you would end up with a neurotic animal - I don't want to do that experiment.


----------



## RedDogs (Jan 30, 2010)

GoldenSail said:


> As far as 'disappointment' maybe I am being lenient, but I have worked with dogs that when you reached out to pet them jerked away because you did not have a cookie for the reward like they expected. This bothers me greatly.


I'll agree that I've not seen anything in the literature about this sort of thing, nor do I beleive it has formally been looked into.

However, with this example... if we were teaching a dog to enjoy petting. We would pet, then get out food and feed. Pet. Feed. Pet. Feed. If good mechanics are being used (only rech for the treat after the pet happens, NOT before, NOT indicate you have food on you (though they can smell it and likely assume after the first rep).... and repeat. And if the dog really is discriminating between food on and food not on the person... there's ways around that.

I don't see your above example as a reason for not creating a stronger association with food, esp for dogs who dislike social contact with people.


----------



## GoldenSail (Dec 30, 2008)

RedDogs said:


> I'll agree that I've not seen anything in the literature about this sort of thing, nor do I beleive it has formally been looked into.
> 
> However, with this example... if we were teaching a dog to enjoy petting. We would pet, then get out food and feed. Pet. Feed. Pet. Feed. If good mechanics are being used (only rech for the treat after the pet happens, NOT before, NOT indicate you have food on you (though they can smell it and likely assume after the first rep).... and repeat. And if the dog really is discriminating between food on and food not on the person... there's ways around that.
> 
> I don't see your above example as a reason for not creating a stronger association with food, esp for dogs who dislike social contact with people.


Sure, but I still think you might create a dog that tolerates petting rather than enjoys petting. Not that it matters a ton, and honestly every dog is different. I have just seen the idea of associating food with praise makes praise more valuable and I don't think that is true and definitely not backed up.

And the argument isn't about whether or not you should do it. To each their own! The question was--does is work? Is it supported? And I don't think there is any proof either way.


----------



## GoldenSail (Dec 30, 2008)

The_Artful_Dodger said:


> I wouldn't consider praise on its own to be a secondary reinforcer...because I think of secondary as being a neutral stimulis that is conditioned to be a reinforcer. If the praise is intrinsically rewarding than I would consider it a primary reinforcer. So I would think what you are asking is that by pairing two primary reinforcers (food and praise)- will the weaker primary reinforcer become strengthened by the stronger primary reinforcer? I dont know but I would think no. But the combined effect of the drive for food and drive for praise when used together might increase the overall motivation to perform the desired behaviour.


Right! So there is a problem. It seems that hard science uses the definition of a secondary as you described, but dog trainers like to use the definition that secondary is praise/play/petting--not necessary for survival. So, it's hard to understand each other and what scientific articles say if we are not using the language in the same fashion, which, it appears, we are not.


----------



## ebenjamin85 (Apr 13, 2008)

Though I'm not an animal behaviorist, I have a great deal of experience and education on human behavior... and I'm taking the chance that we have some similarities here.

A primary reinforcer is one that does not need to be conditioned to be reinforcing, but is naturally reinforcing as it generally meets a basic need... think food, water, sex. No one has to teach you that these things are pleasurable/good, but rather evolution has made them as such.

A secondary reinforcer is one that is conditioned, or learned. However, things with Golden's are likely sticky here as one can argue that affection is a primary reinforcer for the breed (as frequently bred for disposition and they are generally affectionate by nature). 

A secondary reinforcer is thus something that you condition someone (or a dog) to like, think a clicker, a ball, etc. It becomes fun because you have made it enjoyable by previously pairing it with a primary reinforcer. 

When talking about reinforcement think the acronym DISC:

*D*eprivation (to avoid satiation)
*I*mmediacy (following the desired behavior)
*S*ize (enough to reinforce but not enough to satiate)
*C*ontingency (on behavior that you want to increase)

I hope that that makes sense and is truly in-line with the thread.


----------



## ebenjamin85 (Apr 13, 2008)

As an addition to some other things that I've seen in the thread...

Reinforcement increases the frequency of a behavior and is either the addition of a pleasurable stimulus or the removal of an aversive stimulus. 

Punishment decreases the frequency of a behavior and is either the addition of an aversive stimulus of the removal of a pleasurable stimulus.

Variable schedules of reinforcement are always better at maintaining a behavior, but need to be faded into from a continuous schedule of reinforcement carefully.


----------



## tippykayak (Oct 27, 2008)

GoldenSail said:


> As far as 'disappointment' maybe I am being lenient, but I have worked with dogs that when you reached out to pet them jerked away because you did not have a cookie for the reward like they expected. This bothers me greatly.


Sure, but you can't generalize from that behavior. Food was greatly misused with that dog. I once had a thumbnail cracked in half by a dog who fear-bit me when I grabbed for his collar. I believe this was because he was severely mistrained with an e-collar. That doesn't mean that all dogs who are e-collar trained become fear biters or that e-collars necessarily lead to fearful behavior.

Also, if a dog gets food every single time, he's going to look for food first. That doesn't mean that praise isn't reinforcing.


----------



## tippykayak (Oct 27, 2008)

GoldenSail said:


> So, therefore I haven't found anything that supports or refutes the idea that pairing praise with food makes praise better. I personally think it could be a slippery slope, and am doubtful that you could take a dog that dislikes praise/petting and get it to work for those things by pairing it with food.


My personal experience and the basic principle of association supports the idea. I don't know about "proving" it beyond the shadow of a doubt. I know that when my dogs complete a skill, they absolutely live for the "good dog" at the end of it. I believe that part of their intense desire to please comes from their biddability, which is bred to the bone, but I do think that part of the motivation comes from the fact that when we play a game (and all work is _always_ a game), figuring out how to execute what I'm asking is always a blast.

And I think food can be an important part of making those games a blast. I don't use it as my sole reinforcer, and maybe that's why I don't have problems with my dogs being cookie-focused. I do use it a lot during shaping, but by the time the dog has acquired the skill and is simply going through it for me, treats are truly sporadic.

If a dog dislikes praise, it has somehow become aversive. Turning an aversive into a reinforcer would be a ridiculously hard uphill battle, and I don't know why you would. If the dog is uncomfortable being touched, you just shouldn't use touch as a reinforcer in training. It would be like slapping your dog and then giving him a cookie and saying the cookie could make the dog like the slap. Yes, maybe it could eventually do so, but the slap would probably never be a good reinforcer for that dog.

I'm sure it's possible, but I don't think it's practical.


----------



## Megora (Jun 7, 2010)

ebenjamin85 said:


> A secondary reinforcer is thus something that you condition someone (or a dog) to like, think a clicker, a ball, etc. It becomes fun because you have made it enjoyable by previously pairing it with a primary reinforcer.


I couldn't resist the comment that there is a huge difference between using a clicker and a toy as a reinforcer. 

A clicker is a "good sound" because you condition a dog to think that "click means treat". I'm not into clicker training and there are different methods that I've heard of, but the one class I took (and bungled the whole clicker thing) worked out that way. Dog presented a desired behavior, you clicked and treated, dog learned that presenting that behavior got that wonderful clicking sound.

Using the word "YES" instead of click works the same way. That's something I could handle and I've seen it really works the way it is supposed to. Basically the dog hears the same type of sound each time he does something good. That sound marks the behavior so he learns faster. Or something to that effect. My guy loves to hear "yes". 

For that matter "NO" or "Ah-Ah" is a cue you teach your dog if it is combined with a corrective pop. The pop goes away, but you can just go "Ah-Ah" when you notice the dog doing something wrong and get him back on the straight and narrow. <- Case in point, last night we had distractions from a toy bunny on the side of the training floor. I'd heel past and see his eyes twitching and give that "ah-ah" correction, his eyes would fix on me again and we'd glide past without him actually looking at the rabbit. 

With toys - for one thing, goldens naturally are drawn to anything they can stuff in their mouth and carry around. If you didn't play with them, then they'd probably play by themselves. The toy itself _is_ a primary reinforcer and in some cases a stronger one than food. This because you are engaging your dog's prey drive. A retrieve is you allowing your dog to do what he had been bred to do. And of course that breeding evolution took that "chase and kill" instinct and made it something useful to us. <- Not saying my dog doesn't do his best to chase and kill his stuffies. He does do that whole pouncing and shaking to death thing, and given enough time, the shaking to death turns into ripping the toy assunder.


----------



## GoldenSail (Dec 30, 2008)

tippykayak said:


> Sure, but you can't generalize from that behavior. Food was greatly misused with that dog. I once had a thumbnail cracked in half by a dog who fear-bit me when I grabbed for his collar. I believe this was because he was severely mistrained with an e-collar. That doesn't mean that all dogs who are e-collar trained become fear biters or that e-collars necessarily lead to fearful behavior.
> 
> Also, if a dog gets food every single time, he's going to look for food first. That doesn't mean that praise isn't reinforcing.


How do you know that food was misused? Some dogs are not that into praise. Some dogs will take a piece out of you that have had no prior negative experiences. I think anytime you use lots of food in training in comparison to other things you are at risk for creating that dog even if are on a variable schedule and hide the food. I would guess the risk is less severe in breeds like goldens because they are born and bred to be pretty biddable with humans unlike some more independent breeds.


----------



## GoldenSail (Dec 30, 2008)

tippykayak said:


> My personal experience and the basic principle of association supports the idea. I don't know about "proving" it beyond the shadow of a doubt. I know that when my dogs complete a skill, they absolutely live for the "good dog" at the end of it. I believe that part of their intense desire to please comes from their biddability, which is bred to the bone, but I do think that part of the motivation comes from the fact that when we play a game (and all work is _always_ a game), figuring out how to execute what I'm asking is always a blast.
> 
> And I think food can be an important part of making those games a blast. I don't use it as my sole reinforcer, and maybe that's why I don't have problems with my dogs being cookie-focused. I do use it a lot during shaping, but by the time the dog has acquired the skill and is simply going through it for me, treats are truly sporadic.


Yes, but if you are using praise always and food sometimes--then you don't really have a strong association. Praise does not indicate food, so how does food make praise better?


----------



## Loisiana (Jul 29, 2009)

The_Artful_Dodger said:


> Do you mean pairing something adversive to decrease a behaviour with a primary reinforcer? I would call that confusing! I would imagine that if you paired say a shock and a food reward as a response to behaviour, you would end up with a neurotic animal - I don't want to do that experiment.


But isn't that what a lot of dog trainers do? For example, I initially introduce a collar pop by popping the dog towards a treat. After that initial conditioning, collar pops are always following by praise and/or play. When I later incorporate collar pops into my training, if my dog looks away I will give a collar pop and the dog should immediately tune back into me in order to find out what kind of game we're about to play.


----------



## GoldenSail (Dec 30, 2008)

Anyway, no need for this thread to keep continuing really. I have just heard a few times that praise (secondary) is strengthened when associated with food (primary) and I was skeptical. I was told that lots of stuff backs up this idea, but not given specific articles. So, when I did my own search I found that secondaries as defined by those conditioned with a primary did become strengthened with a primary--but using that definition it is part of the assumption! So, nothing that actually made a connection between something like praise and food.

And I am really not that into the science, I like thinking of dog training as more of an art form than anything else. But, I do try to understand it because I think a rough understanding is important. 

(I remain dubious that the purposeful association with food and praise will make praise more valuable)

Anyway, so happy there are so many different ways to train a dog--makes it more fun and interesting!


----------



## The_Artful_Dodger (Mar 26, 2009)

Loisiana said:


> But isn't that what a lot of dog trainers do? For example, I initially introduce a collar pop by popping the dog towards a treat. After that initial conditioning, collar pops are always following by praise and/or play. When I later incorporate collar pops into my training, if my dog looks away I will give a collar pop and the dog should immediately tune back into me in order to find out what kind of game we're about to play.


Interesting question. I'm not an expert dog trainer - I think I'm pretty good compared to regular people - but definitely a novice in the obedience world. So my anwer is based on psychology and not dog-training-knowledge. Here's what I think....and this could be completely wrong...just thinking here

The collar pop may have initially been adversive but the dog can become desensitized to it. Going back to my rat example. When I first started training her, we had to desensitize her to the clicking sound, which initially produced a fear response. We extingished the fear response and she learned to associate the clicking sound with food. I guess you can turn something adversive into something neutral by desensitizing them to it (depending on the severity or the adversive)


Ok...so now we have the dog desensitized to the collar pop. Then they are taught that following a collar pop if they focus they get rewarded (with food or praise or whatever). Focusing behaviour increases following collar pops. You are using a collar pop as a secondary reinforcer. 

(I'm not saying collar pops are always a reinforcer - that's just how I see it with this particular example)


----------



## The_Artful_Dodger (Mar 26, 2009)

The_Artful_Dodger said:


> You are using a collar pop as a secondary reinforcer.
> 
> (I'm not saying collar pops are always a reinforcer - that's just how I see it with this particular example)


Changed my mind - since the collar pop is coming before the behaviour, its not a reinforcer. I think you are using a collar pop as a discriminatory stimulus (or a cue). 

Now here is an interesting question - if the dog learns that a collar pop signals that if they focus at that time they will be rewarded - then will the behaviours that are associated with the collar pop be reinforced? That is, if every time you collar pop and the dog looks at you you give a reward, and then you collar pop when the dog is say looking away - then will the looking away beaviour become more frequent so they can get the collar pop and therefore be able to get rewarded for focusing. Then the collar pop would be a secondary reinforcer but would be rewarding the looking away behaviour


----------



## Megora (Jun 7, 2010)

> then will the looking away beaviour become more frequent so they can get the collar pop and therefore be able to get rewarded for focusing.


That will happen if you nag and not following through with marking the good behavior. 

Pops on the collar are either corrective or they are specific cues that go away as soon as you teach your dog to focus using your voice or body language alone. That pop correction strengthens those other corrections, about the same way the treats and toys strengthen praise and play. 

So if somebody was using a clicker type method, the click would come AS SOON as the dog looked back at the trainer. 

The difference is that the trainer isn't waiting for the dog to decide when to look up. They are doing the pop correction to instigate the good behavior.


----------



## Loisiana (Jul 29, 2009)

Just a few more notes on the collar pop example. The collar pop used in this manner shouldn't be something you have to desensitize (unless you just have a really sensitive dog), it should be something the dog likes from day one. Kind of like if someone came and slapped my hand I might get a little ticked off and even complain that I didn't like that but if someone came and smacked a twenty dollar bill in my hand with the same amount of force I would probably think I kind of like that feeling. Same feeling to my hand, but done with different initial associations would give me different attitude towards how it felt. So if a dog is just popped without the reward then he would need to be desensitized but if you treat it as a game from day one then they should enjoy it from day one.

The pop is ALWAYS followed by praise or play, even if the dog doesn't immediately focus back. I'm basically praising the pop, not the attention. The hope is just that the result of that is better attention.

The area that's always confused me about reinforcers and punishers is when do you classify it as decreasing a behavior and when do you classify it as increasing a behavior. If I pop when the dog looks away in order to get gain his attention, am I decreasing the look away or increasing the focus on me? If I initiate the pop when he looks away then wouldn't that be decreasing the look away, which would make it a punisher? Even if the dog enjoys the punisher, that's still technically what it is isn't it? Maybe a secondary positive punisher with positive associations?


----------



## RedDogs (Jan 30, 2010)

Loisiana said:


> The area that's always confused me about reinforcers and punishers is when do you classify it as decreasing a behavior and when do you classify it as increasing a behavior. If I pop when the dog looks away in order to get gain his attention, am I decreasing the look away or increasing the focus on me? If I initiate the pop when he looks away then wouldn't that be decreasing the look away, which would make it a punisher? Even if the dog enjoys the punisher, that's still technically what it is isn't it? Maybe a secondary positive punisher with positive associations?


It is confusing and complicated! 

Positive punishment and negative reinforcement go hand in hand, as do positive reinforcement and negative punishment... in those pairings it's very hard (if not impossible?) to use one without the other.

Whether you label it one way or another, depends on which behavior you are looking at. So if you are specifically talking about looking away, you would say the pop is a P+, if you are specifically looking at the attention to you, it would be talked about R-. 

IF looking away increased, to get the pop (and play), then you would talk about the pops as R+. 

But as the looking typically decreases over time, evne though the pops are followed by play/praise/etc...., the increase in attention would be discussed as a result of R-, the dog is working to avoid the leash pops.


----------



## The_Artful_Dodger (Mar 26, 2009)

Loisiana said:


> The pop is ALWAYS followed by praise or play, even if the dog doesn't immediately focus back. I'm basically praising the pop, not the attention. The hope is just that the result of that is better attention.
> 
> The area that's always confused me about reinforcers and punishers is when do you classify it as decreasing a behavior and when do you classify it as increasing a behavior. If I pop when the dog looks away in order to get gain his attention, am I decreasing the look away or increasing the focus on me? If I initiate the pop when he looks away then wouldn't that be decreasing the look away, which would make it a punisher? Even if the dog enjoys the punisher, that's still technically what it is isn't it? Maybe a secondary positive punisher with positive associations?


 
If you praise/play following the collar pop no matter what the dog does - then what are you trying to teach? The praise/play may direct the dogs attention to you and therefore they may look at you because that is more pleasurable than what they were doing...but it seems like if the collar pop is associated with praise/play than it should reinforce the behaviour they were doing at the time of the collar pop. 

If you are looking to decrease a behaviour (e.g. looking away) by introducing an incompatible behaviour (e.g., looking at you) and rewarding the looking-at-you behaviour - assuming the outcome is that the looking-at-you beaviour increases than it is positive reinforcement. The looking away behaviour has decreased....but only because the dog can't do both at the same time. 

If you decrease a behaviour through positive punishment - say a pop on a choke chain (assuming the dog finds that adversive) makes looking-away behaviour less likely. This also makes looking-at-you beaviour more frequent because if the dog is not looking away from you then they have to be looking at you. 

I think it's just confusing in this context because there are only two options - looking at you or looking away - so one has to decrease if the other increases. If say you use an adversive only when the dog sniffed at the ground - then sniffing the ground behaviour would likely decrease but that wouldn't necessariliy make looking-at-you behaviour increase by much because there are other behaviours they can do. But if you leash pop and reward every time they sniff the ground - if they don't find the leash pop adversive - I would imagine that sniffing the ground would become more frequent


----------



## Loisiana (Jul 29, 2009)

Actually it works great for sniffing too. Pop and play everytime a dog sniffs and sniffing will be greatly reduced and attention increased.

That's why I don't get very wrapped up in the science when dog training - there's too much that I do that WORKS, and I don't understand why it works but it does, over and over with many different dogs. I'm sure if I thought hard enough and long enough and did enough research I could figure out why but for me it's not as important why as just the fact that it does work.


----------



## Loisiana (Jul 29, 2009)

Okay I thought about it on the way home. Here is why I think it works (in non-scientific terms).

Imagine that everytime you didn't pay attention to a particular person, they came up and shoved you and either didn't say anything or yelled at you "you better always pay attention to me!" Most likely you would either pay more attention to that person out of fear of the consequences, or you would start to melt down and go into avoidance.

On the other hand, imagine that everytime you didn't pay attention to a particular person, they came up and shoved you and said in a fun teasing voice "ha ha, gotcha!" Now shove isn't seen as threatening, it's seen as a game. You're going to do your best to not lose focus on that person because you want to win. If they get their shove in then they're winning. It's all in good fun.

I really think the dogs see it like that - it's all a game. If they look away then I get my shove/pop/poke/whatever in. But if I do, oh well, just they'll just pay more attention next time so they can win. It's not seen as anything threatening.

Taking this even further from the topic, I think that's why a lot of people are so against the use of corrections in training. They think of using corrections as the first person in my example, while in reality the majority of GOOD trainers that I know train like the second person. It's not about intimidating or forcing the dog, it all fun and happy.


----------



## tippykayak (Oct 27, 2008)

GoldenSail said:


> How do you know that food was misused? Some dogs are not that into praise. Some dogs will take a piece out of you that have had no prior negative experiences. I think anytime you use lots of food in training in comparison to other things you are at risk for creating that dog even if are on a variable schedule and hide the food. I would guess the risk is less severe in breeds like goldens because they are born and bred to be pretty biddable with humans unlike some more independent breeds.


I was assuming it was misused based on the scenario you described. I thought you were saying that the dog had learned to ignore praise because of the food.


----------



## tippykayak (Oct 27, 2008)

GoldenSail said:


> Yes, but if you are using praise always and food sometimes--then you don't really have a strong association. Praise does not indicate food, so how does food make praise better?


Because sporadic reinforcement creates more durable associations than regular reinforcement. So doing it sporadically makes it even stronger than doing it every time.


----------



## ebenjamin85 (Apr 13, 2008)

Megora said:


> I couldn't resist the comment that there is a huge difference between using a clicker and a toy as a reinforcer.


Very true. My point was that they are both secondary reinforcers because they both become reinforcing through teaching and are not naturally reinforcing (like a primary reinforcer).


----------



## FlyingQuizini (Oct 24, 2006)

GoldenSail said:


> How do you know that food was misused? Some dogs are not that into praise. Some dogs will take a piece out of you that have had no prior negative experiences. *I think anytime you use lots of food in training in comparison to other things you are at risk for creating that dog even if are on a variable schedule and hide the food.* I would guess the risk is less severe in breeds like goldens because they are born and bred to be pretty biddable with humans unlike some more independent breeds.


OK. Since I know you're a fan of studies... can you cite studies that support this theory? 

I've hated to be so MIA from this conversation. School started and I've been slammed. I don't have time to go looking for cited studies, but I'll email a group of trainer friends and see what they have.

I admit... I'm still stumped as to how you can feel so strongly about food creating a problem, even when used "correctly" (let's loosely define "correctly" as well-timed, reward v. lure etc.). I mean, the idea of strengthening praise with food is, at its core, classical conditioning. Do you doubt the effectiveness of classical conditioning? I'm hearing dear, sweet Bob Bailey in my ear as I type this... "Pavlov is always sitting on your shoulder..."



> I would guess the risk is less severe in breeds like goldens because they are born and bred to be pretty biddable with humans unlike some more independent breeds.


OR..... Pavlov is always sitting on your shoulder and there really is some truth to primaries strengthening secondaries! : My first obedience dog was my Whippet... a female... HOUND... Not a a breed known for being biddable... but that IS known for being independent.

Dogs make classically conditioned associations all the time... the dog that goes crazy when they see you pick up the leash, or say "w.a.l.k," or when clanging the empty bowl brings the dog flying in from another room. One is strengthening the other. Even if we forget about primary v. secondary... it's one strengthening the other.


----------



## GoldenSail (Dec 30, 2008)

FlyingQuizini said:


> OK. Since I know you're a fan of studies... can you cite studies that support this theory?
> .


I said I *think* meaning that is an opinion. I did not assert that science was on my side. Pet peeve of mine is when people say their idea is supported by science but can't direct you to articles that support the idea. If it is an opinion--great! But if it is science better be able to back it up, JMO.



> I admit... I'm still stumped as to how you can feel so strongly about food creating a problem, even when used "correctly" (let's loosely define "correctly" as well-timed, reward v. lure etc.). I mean, the idea of strengthening praise with food is, at its core, classical conditioning. Do you doubt the effectiveness of classical conditioning? I'm hearing dear, sweet Bob Bailey in my ear as I type this... "Pavlov is always sitting on your shoulder..."


Probably because who gets to define correct? I mean seriously? If a dog becomes dependent on food does that automatically mean it was used incorrectly? Even if the person is using variable reinforcement, hiding the food, etc etc? I've read a lot on another list about many good trainers (OTCH level even) having a fallout with food.

Yes I realize praise/food as pavlov. But, what I am getting at is if you are not associating food with praise always (since you are variable with food), the connection you think or want to make might not be there. If pavlov only presented food sometimes when he rang the bell, but other times just rang the bell at *some point* I bet the dogs would not correlate food with the bell. And as mentioned before, I don't think I know anyone that uses a clicker as a stand alone reinforcer. It would be interesting to see how long it would take for a clicker to lose its value when not associated with food.

Also, what remains to be studied is the effect of using two reinforcers together. I know that if I went to work and got praised at the end of the day and a paycheck I would be very happy. If I was classically conditioned to associated praise with paycheck, then the day that I got praise with no paycheck I would be disappointed. And when it was on a variable schedule, I would come to start to lose the association with being praised at work and getting a paycheck since the praise does not always end in getting a paycheck. Not only that, but since the paycheck is a lot more valuable than praise I think I would tend to ignore or not care about the praise--just waiting and wanting the paycheck. Anyway, kind of trying to explain my though process.


----------



## GoldenSail (Dec 30, 2008)

FlyingQuizini said:


> My first obedience dog was my Whippet... a female... HOUND... Not a a breed known for being biddable... but that IS known for being independent.


Yes and you stopped at the CD--why? How does she compare to Quiz?


----------



## FlyingQuizini (Oct 24, 2006)

GoldenSail said:


> I said I *think* meaning that is an opinion. I did not assert that science was on my side. Pet peeve of mine is when people say their idea is supported by science but can't direct you to articles that support the idea. If it is an opinion--great! But if it is science better be able to back it up, JMO.
> 
> 
> 
> ...


I guess it's a matter of what you're most frequently exposed to. I can think of many OTCh trainers who have great success (w/o the fallout) with food.

The difference between the specifics of the Pavlov experiment and what we're talking about is that the bell is so, so very neutral, whereas we both agree that praise is rewarding, thanks to the wonderful ability of well-adjusted dogs to form relationships with humans. (I still consider it a secondary reinforcer - dogs aren't BORN liking praise and play with humans, but it's something they learn VERY fast.) I'm just saying that I strongly suspect classical conditioning is going on in the background when I choose to pair food with praise/petting. I don't think that if I were to suddenly stop using food, that eventually my praise/petting, etc. would lose value -- b/c it was always somewhat rewarding; I just tried to make it better. 

I'm struggling with your example of praise and paycheck b/c in that example, both are secondary reinforcers. A paycheck it just a piece of paper. It only has value b/c we've learned it can be used to get good stuff.

With the Pavlov thing... yes, if food only came intermittently when he rang the bell, the bell would lose power... but again, I think that's b/c the bell started out as a nothing. In our example - at least my version of it - I'm saying my "bell" (the praise/petting) started out with some power b/c of the ability to form relationships. I'm simply making the bell better by adding the meat!

Would my bell (praise/petting/play) lose value over time if I stopped using food starting tomorrow? I don't know. There's such a strong history there, I think it would take a long time. 

As for use of food creating "don't touch me dogs" I see that happen often - but oddly enough, it seems to ONLY be in the people who are Pez Dispensers when they reward with food. As soon as I get them petting THEN feeding, the duck-away behavior goes away in the majority of them. Does that mean the dog now "enjoys" the petting? Well, they aren't talking - but if I had to judge by body language, I'd say some yes, some no. But even that has so many variables -- such as HOW the dog is being petted. 

I'm not even sure how you'd set up a study to look at the potential ability of food to raise the value of petting. I'm sure there's some hormone you could look at, but how would you isolate whether or not the increase was due to to the pairing or just a function of the dog consuming something necessary for survival?


----------



## FlyingQuizini (Oct 24, 2006)

GoldenSail said:


> Yes and you stopped at the CD--why? How does she compare to Quiz?


They are worlds apart. Quiz has drive that goes for days. Zoie has about as much natural drive as a Yugo! :

Quiz enjoys working and will happily work for a number of reinforcers: food, toys, interaction, etc. Zoie is not nearly as motivated by any of those things. She simply didn't have as much joy for the work - regardless of the reinforcer - so it seemed kinder to let her retire.


----------



## GoldenSail (Dec 30, 2008)

I guess I could see it depending on how you are associating praise and food--if you are using a marker great. I can see it. But, dogs are individuals not machines! You cannot apply science to them all the same way and expect the same results.

And I see what you mean using two secondaries--lol but I work better for money than I do food. What's up with that?  Associating food with the money might make the work that much better, but it wouldn't make the food better for me and ultimately if I had been trained that I get paid money for my work, that's what I want. Now, if I was taught to expect something lesser of value (like a cupcake, lol) ok, I would probably be happy working for that as long as I did no have a strong history of getting something better 

Earlier in the thread too we mentioned the rats that were trained by being tickled. How cool! I think it would be interested to do an experiment with rats first (keep it cleaner than with dogs) and use tickling as the so-called secondary (but is it?? it certainly does not seem equatable to a conditioned clicker). Have a group that is only tickled for the right response, have a group that is tickled then fed, have a group that is fed then tickled, and have a group that is only fed. After the tickle/food association has been made, put the rats on a primary schedule with the food, but continuous with the tickling. Then try to measure their responses. Do they act differently when food is not presented when they expect it? Which group will last longer if only tickling is being used?


----------

