Behavioral and associative effects of differential outcomes in discrimination learning

The role of the reinforcer in instrumental discriminations has often been viewed as that of facilitating associative learning between a reinforced response and the discriminative stimulus that occasions it. The differential-outcome paradigm introduced by Trapold (1970), however, has provided compelling evidence that reinforcers are also part of what is learned in discrimination tasks. Specifically, when the availability of different reinforcing outcomes is signaled by different discriminative stimuli, the conditioned anticipation of those outcomes can provide another source of stimulus control over responding. This article reviews how such control develops and how it can be revealed, its impact on behavior, and different possible mechanisms that could mediate the behavioral effects. The main conclusion is that differential-outcome effects are almost entirely explicable in terms of the cue properties of outcome expectancies—namely, that conditioned expectancies acquire discriminative control just like any other discriminative or conditional stimulus in instrumental learning.

[1]  J. Bruce Overmier,et al.  Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific , 1983 .

[2]  M. Papini,et al.  Mechanisms of acquisition of behavioral sequences in rats , 1989 .

[3]  T. Davidson Hunger cues as modulatory stimuli. , 1998 .

[4]  E. Wasserman,et al.  Non-Similarity-Based Conceptualization in Pigeons via Secondary or Mediated Generalization , 1992 .

[5]  K. Haberlandt,et al.  Stimulus selection in animal discrimination learning. , 1968, Journal of Experimental Psychology.

[6]  G. Peterson,et al.  Enhancement of pigeons’ conditional discrimination performance by expectancies of reinforcement and nonreinforcement , 1980 .

[7]  H. M. Jenkins,et al.  The form of the auto-shaped response with food or water reinforcers. , 1973, Journal of the experimental analysis of behavior.

[8]  M. A. Trapold,et al.  Are expectancies based upon different positive reinforcing events discriminably different , 1970 .

[9]  G. Peterson,et al.  Two-choice conditional discrimination performance of pigeons as a function of reward expectancy, prechoice delay, and domesticity , 1976, Animal learning & behavior.

[10]  K. White,et al.  On two effects of signaling the consequences for remembering , 1995 .

[11]  T. Zentall Symbolic representation in animals: Emergent stimulus relations in conditional discrimination learning , 1998 .

[12]  L. Savage,et al.  The effects of delay interval, intertriai interval, amnestic drugs, and differential outcomes on matching-to-position in rats , 1997, Psychobiology.

[13]  J. Overmier,et al.  Behavioral and pharmacological analyses of memory: New behavioral options for remediation. , 1999 .

[14]  Colleen Reichmuth,et al.  How Animals Classify Friends and Foes , 2000 .

[15]  P. Urcuioli Differential outcomes and many-to-one matching: Effects of correlation with correct choice , 1990 .

[16]  J. G. Carlson,et al.  Delay of reinforcement in instrumental discrimination learning of rats. , 1972, Journal of comparative and physiological psychology.

[17]  J. Deese The psychology of learning , 1952 .

[18]  R. Rescorla Transfer of instrumental control mediated by a devalued outcome , 1994 .

[19]  G. Peterson,et al.  Effects of altering outcome expectancies on pigeons' delayed conditional discrimination performance , 1980 .

[20]  Luis J. Fuentes,et al.  Differential outcomes effect in children: Demonstration and mechanisms ☆ , 2003 .

[21]  Prospective representation: The effects of varied mapping of sample stimuli to comparison stimuli and differential trial outcomes on pigeons’ working memory , 1985 .

[22]  H. M. Jenkins,et al.  Overshadowing in operant conditioning as a function of discriminability , 1973 .

[23]  R. Colwill Associative Representations of Instrumental Contingencies , 1994 .

[24]  A. Goodwin,et al.  An analysis of the utility of differential outcome procedures in drug discrimination research , 2002, Behavioural pharmacology.

[25]  R. Rescorla,et al.  Evidence for the hierarchical structure of instrumental learning , 1990 .

[26]  W. V. Dube,et al.  Stimulus class membership established via stimulus-reinforcer relations. , 1987, Journal of the experimental analysis of behavior.

[27]  M. Markham,et al.  Stimulus equivalence, functional equivalence and the transfer of function. , 1994 .

[28]  A Differential Outcomes Effect using Biologically Neutral Outcomes in Delayed Matching-to-Sample with Pigeons , 2001, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[29]  A. Hochhalter,et al.  Differential outcomes training facilitates memory in people with Korsakoff and Prader-Willi syndromes , 2001, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[30]  L. Baker,et al.  The application of differential outcomes to a cocaine—saline discrimination procedure , 1997, Behavioural pharmacology.

[31]  Asratyan Ea Conditional reflex theory and motivational behavior. , 1974 .

[32]  C. L. Hull Knowledge and purpose as habit mechanisms. , 1930 .

[33]  T. DeMarse,et al.  Transfer of performance to new comparison choices following differential outcome matching-to-sample , 1998 .

[34]  W A Roberts,et al.  Further evidence for hierarchical chunking in rat spatial memory. , 1995, Journal of experimental psychology. Animal behavior processes.

[35]  W. V. Dube,et al.  Stimulus-Reinforcer Relations and Emergent Matching to Sample , 1995 .

[36]  R. Schusterman,et al.  Equivalence classification by California sea lions using class-specific reinforcers. , 2001, Journal of the experimental analysis of behavior.

[37]  J. G. Carlson,et al.  Mediators of the effects of magnitude of reinforcement , 1976 .

[38]  J. Overmier,et al.  Pavlovian Conditioning and the Mediation of behavior1 , 1979 .

[39]  B. Skinner,et al.  Principles of Behavior , 1944 .

[40]  T R Zentall,et al.  Episodic-like memory in pigeons , 2001, Psychonomic bulletin & review.

[41]  W. Sailor,et al.  A Comparison of Three Strategies of Reinforcement on Two-Choice Learning Problems with Severely Retarded Children , 1979 .

[42]  G. Peterson,et al.  On the comparative efficacy of memories and expectancies as cues for choice behavior in pigeons , 1987 .

[43]  Karen L. Chambers,et al.  Differential Outcomes Effect: Increased Accuracy in Adults Learning Kanji with Stimulus Specific Rewards , 2002 .

[44]  Role of differential sample responding in the differential outcomes effect involving delayed matching by pigeons. , 1994, Journal of experimental psychology. Animal behavior processes.

[45]  N. Schmajuk,et al.  Occasion setting: Associative learning and cognition in animals. , 1998 .

[46]  T. DeMarse,et al.  Assessing the contributions of S-O and R-O associations to differential-outcome matching through outcome reversals. , 2001, Journal of experimental psychology. Animal behavior processes.

[47]  I. Gormezano,et al.  The pavlovian analysis of instrumental conditioning , 1976, The Pavlovian journal of biological science.

[48]  D. Crowne,et al.  Experimental foundations of clinical psychology , 1964 .

[49]  C. L. Hull,et al.  A Behavior System , 1954 .

[50]  T. Zentall,et al.  Transfer across delayed discriminations: II. Differences in the substitutability of initial versus test stimuli. , 1998, Journal of experimental psychology. Animal behavior processes.

[51]  T. DeMarse,et al.  On the relationship between differential outcomes and differential sample responding in matching-to-sample , 1994 .

[52]  K. White,et al.  An investigation of the differential-outcomes effect within sessions. , 1994, Journal of the experimental analysis of behavior.

[53]  J. D. de Rose,et al.  Emergent simple discrimination established by indirect relation to differential consequences. , 1988, Journal of the experimental analysis of behavior.

[54]  R. Rescorla,et al.  Associative Structures In Instrumental Learning , 1986 .

[55]  Overmier Jb,et al.  Discriminative cue properties of different fears and their role in response selection in dogs. , 1971 .

[56]  Stewart H. Hulse Psychology of Learning , 1980 .

[57]  J. Bruce Overmier,et al.  Expectancies as Factors Influencing Conditional Discrimination Performance of Children , 1995 .

[58]  E. Wasserman,et al.  Effects of differential reinforcement expectancies on successive matching-to-sample performance in pigeons. , 1981, Journal of experimental psychology. Animal behavior processes.

[59]  R. C. Honey,et al.  Acquired equivalence and distinctiveness of cues. , 1989, Journal of experimental psychology. Animal behavior processes.

[60]  E. Wasserman,et al.  Superordinate categorization via learned stimulus equivalence: quantity of reinforcement, hedonic value, and the nature of the mediator. , 2001, Journal of experimental psychology. Animal behavior processes.

[61]  The differential outcomes procedure can interfere or enhance operant rule learning , 2003, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[62]  P. Urcuioli 4 Acquired equivalences and mediated generalization in pigeon's matching-to-sample , 1996 .

[63]  W. K. Honig,et al.  The legacy of Guttman and Kalish (1956): Twenty-five years of research on stimulus generalization. , 1981, Journal of the experimental analysis of behavior.

[64]  W. V. Dube,et al.  Stimulus class formation and stimulus-reinforcer relations. , 1989, Journal of the experimental analysis of behavior.

[65]  E. Tolman Sign-Gestalt or conditioned reflex. , 1933 .

[66]  P. Urcuioli,et al.  Some tests of the anticipatory mediated generalization model of acquired sample equivalence in pigeons’ many-to-one matching , 2001 .

[67]  R. Bolles,et al.  Differential outcome effect using a biologically neutral outcome difference. , 1986 .

[68]  Charles A. Edwards,et al.  Acquired equivalence and distinctiveness in matching to sample by pigeons: Mediation by reinforcer-specific expectancies. , 1982 .

[69]  M. Litt,et al.  Stimulus-specific reinforcement in the acquisition of receptive labels by autistic children , 1981 .

[70]  C. L. Hull The problem of stimulus equivalence in behavior theory. , 1939 .

[71]  B. Balleine,et al.  Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain. , 2003, Journal of experimental psychology. Animal behavior processes.

[72]  T. DeMarse,et al.  Enhancement of matching acquisition by differential comparison-outcome associations. , 1993 .

[73]  E. Tolman Purposive behavior in animals and men , 1932 .

[74]  A. Poling,et al.  The effects of differential and nondifferential outcomes on response rates and accuracy under a delayed-matching-to-sample procedure , 1991 .

[75]  L. Savage,et al.  Memory enhancement in aged rats: the differential outcomes effect. , 1999, Developmental psychobiology.

[76]  Effects of a stimulus correlated with positive reinforcement upon discrimination learning , 1973 .

[77]  T. DeMarse,et al.  Further Tests of Response-Outcome Associations in Differential- Outcome Matching-to-Sample , 1997 .

[78]  S. Nakajima,et al.  Differential outcome effect in the horse. , 2000, Journal of the experimental analysis of behavior.

[79]  B. Shepp Some cue properties of anticipated rewards in discrimination learning of retardates. , 1962, Journal of comparative and physiological psychology.

[80]  R. Rescorla,et al.  Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. , 1967, Psychological review.

[81]  J. Overmier,et al.  Expectancies of reinforcer location and quality as cues for a conditional discrimination in pigeons. , 1990, Journal of experimental psychology. Animal behavior processes.

[82]  L. Savage In search of the neurobiological underpinnings of the differential outcomes effect , 2001, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[83]  G. Peterson,et al.  Expectancies as mediators in the differential-reward conditional discrimination performance of pigeons , 1978 .

[84]  P. Urcuioli Retardation and facilitation of matching acquisition by differential outcomes , 1991 .

[85]  J. Schenk Emergent Relations of Equivalence Generated by Outcome-Specific Consequences in Conditional Discrimination , 1994 .

[86]  M. Sidman,et al.  Equivalence relations and the reinforcement contingency. , 2000, Journal of the experimental analysis of behavior.

[87]  T. Zentall,et al.  On the role of trial outcomes in delayed discriminations , 1990 .

[88]  Peter J. Urcuioli,et al.  Some relationships between outcome expectancies and sample stimuli in pigeons’ delayed matching , 1990 .

[89]  R. Rescorla,et al.  Postconditioning devaluation of a reinforcer affects instrumental responding. , 1985 .

[90]  J. Donahoe,et al.  Behavior analysis and revaluation. , 2000, Journal of the experimental analysis of behavior.

[91]  The differential outcomes effect in pigeons is not reduced by eliminating response-outcome associations: Support for a two-process account , 1998 .

[92]  G B Peterson,et al.  Expectancy mediation of concurrent conditional discriminations. , 1982, The American journal of psychology.

[93]  J. Overmier,et al.  Conditional choice-unique outcomes establish expectancies that mediate choice behavior , 2001, Integrative physiological and behavioral science : the official journal of the Pavlovian Society.

[94]  R. Rescorla,et al.  Associations with anticipated and obtained outcomes in instrumental learning , 1989 .

[95]  A. Dickinson,et al.  The Interaction between Discriminative Stimuli and Outcomes during Instrumental Learning , 2003, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[96]  G. Hall,et al.  Lesions of the Basolateral Amygdala Disrupt Selective Aspects of Reinforcer Representation in Rats , 2001, The Journal of Neuroscience.

[97]  W. K. Honig,et al.  Control of choice in conditional discriminations by sample-specific behaviors. , 1980, Journal of experimental psychology. Animal behavior processes.

[98]  R. Rescorla Response-outcome versus outcome-response associations in instrumental learning , 1992 .

[99]  T. Zentall,et al.  Transfer across delayed discriminations: evidence regarding the nature of prospective working memory. , 1992, Journal of experimental psychology. Animal behavior processes.

[100]  R. Rescorla,et al.  Associations between the discriminative stimulus and the reinforcer in instrumental learning. , 1988 .

[101]  B. Shepp SOME CUE PROPERTIES OF REWARDS IN SIMULTANEOUS OBJECT-DISCRIMINATIONS OF RETARDATES. , 1964, Child development.

[102]  Superordinate category formation in pigeons: association with a common delay or probability of food reinforcement makes perceptually dissimilar stimuli functionally equivalent. , 1999, Journal of experimental psychology. Animal behavior processes.

[103]  E. Meehan,et al.  Class-consistent Differential Reinforcement And Stimulus Class Formation In Pigeons. , 1999, Journal of the experimental analysis of behavior.

[104]  Pigeons transfer between conditional discriminations with differential outcomes in the absence of differential-sample-responding cues , 1995 .

[105]  J. Overmier,et al.  Differential outcomes effect in children and adults with down syndrome. , 2003, American journal of mental retardation : AJMR.

[106]  Jacques Vauclair,et al.  Functional categorization of objects and of their pictures in baboons , 1998 .

[107]  S. Nakajima,et al.  Differential Outcomes Effect on Instrumental Serial Feature-Ambiguous Discrimination in Rats , 2000 .

[108]  P. Urcuioli Overshadowing in matching-to-sample: Reduction in sample-stimulus control by differential sample behaviors , 1984 .

[109]  T. DeMarse,et al.  Associative processes in differential outcome discriminations. , 1996 .

[110]  A. Poling,et al.  The effects of phenobarbital on responding under delayed-matching-to-sample procedures with differential and nondifferential outcomes , 1991, Pharmacology Biochemistry and Behavior.

[111]  A. Poling,et al.  Letter recognition by adults with mental handicaps: Improving performance through differential outcomes. , 1992 .

[112]  B. Skinner,et al.  Science and human behavior , 1953 .

[113]  H. M. Jenkins Measurement of stimulus control during discriminative operant conditioning. , 1965, Psychological bulletin.

[114]  W. K. Honig,et al.  Outcome expectancies as mediators for discriminative responding. , 1984 .

[115]  W. Roberts,et al.  Reinforcement expectancy and trial spacing effects in delayed matching-to-sample by pigeons , 1985 .

[116]  J. Overmier,et al.  Improving Face Recognition in Alcohol Dementia , 2001 .

[117]  Geoffrey Hall,et al.  Learning about associatively activated stimulus representations: Implications for acquired equivalence and perceptual learning , 1996 .

[118]  K. Spence Behavior Theory and Conditioning , 1978 .

[119]  B. Joseph,et al.  Food- and nonfood-related differential outcomes in equivalence learning by adults with Prader-Willi syndrome. , 1997, American journal of mental retardation : AJMR.

[120]  Murray Sidman,et al.  Equivalence Relations and Behavior: A Research Story , 1994 .

[121]  J. Spradlin,et al.  Establishing a conditional discrimination without direct training: a study of transfer with retarded adolescents. , 1973, American journal of mental deficiency.

[122]  L. Savage,et al.  Rats exposed to acute pyrithiamine-induced thiamine deficiency are more sensitive to the amnestic effects of scopolamine and MK-801: examination of working memory, response selection, and reinforcement contingencies , 1999, Behavioural Brain Research.

[123]  C. L. Hull Goal attraction and directing ideas conceived as habit phenomena. , 1931 .

[124]  E. Thorndike Animal Intelligence; Experimental Studies , 2009 .