Behavior analysis and revaluation.

Revaluation refers to phenomena in which the strength of an operant is altered by reinforcer-related manipulations that take place outside the conditioning situation in which the operant was selected. As an example, if lever pressing is acquired using food as a reinforcer and food is later paired with an aversive stimulus, the frequency of lever pressing decreases when subsequently tested. Associationist psychology infers from such findings that conditioning produces a response-outcome (i.e., reinforcer) association and that the operant decreased in strength because pairing the reinforcer with the aversive stimulus changed the value of the outcome. Here, we present an approach to the interpretation of these and related findings that employs neural network simulations grounded in the experimental analysis of behavior and neuroscience. In so doing, we address some general issues regarding the relations among behavior analysis, neuroscience, and associationism.

[1]  C. L. Hull.  Knowledge and purpose as habit mechanisms. , 1930 .

[2]  C. L. Hull.  Goal attraction and directing ideas conceived as habit phenomena. , 1931 .

[3]  B. Skinner.  Two Types of Conditioned Reflex and a Pseudo Type , 1935 .

[4]  J. Konorski,et al.  On Two Types of Conditioned Reflex , 1937 .

[5]  B. Skinner,et al.  Are theories of learning necessary? , 2013, Psychological review.

[6]  B. Skinner,et al.  Science and human behavior , 1953 .

[7]  C. Hempel,et al.  Aspects of Scientific Explanation and Other Essays in the Philosophy of Science. , 1966 .

[8]  J. E. Mazur,et al.  Learning and Behavior , 1966 .

[9]  B. Skinner.  What is the experimental analysis of behavior? , 1966, Journal of the experimental analysis of behavior.

[10]  R. Rescorla,et al.  Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. , 1967, Psychological review.

[11]  R. Kovner,et al.  Cue properties vs palatability of flavors in avoidance learning , 1970 .

[12]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[13]  R. Rescorla,et al.  Reinstatement of fear to an extinguished conditioned stimulus. , 1975, Journal of experimental psychology. Animal behavior processes.

[14]  R. Rescorla,et al.  The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. , 1975, Journal of experimental psychology. Animal behavior processes.

[15]  John R. Anderson.  Arguments concerning representations for mental imagery. , 1978 .

[16]  E. Fantino,et al.  Effects on choice of reinforcement delay and conditioned reinforcement. , 1978, Journal of the experimental analysis of behavior.

[17]  J. Overmier,et al.  Pavlovian Conditioning and the Mediation of behavior1 , 1979 .

[18]  W. K. Honig,et al.  Cognitive Processes in Animal Behavior , 1979 .

[19]  P. Chance.  Learning and Behavior , 1979 .

[20]  J. Fuster.  Prefrontal Cortex , 2018 .

[21]  J. Michael.  Distinguishing between discriminative and motivational functions of stimuli. , 1982, Journal of the experimental analysis of behavior.

[22]  David M. Sargent,et al.  The order of presentation of conditioned stimuli during extinction , 1983 .

[23]  R. Rescorla,et al.  Postconditioning devaluation of a reinforcer affects instrumental responding. , 1985 .

[24]  B. Skinner.  Is it behaviorism? , 1986, Behavioral and Brain Sciences.

[25]  B. Williams.  Identifying Behaviorism’s Prototype: A Review of Behaviorism: A Conceptual Reconstruction by G. E. Zuriff , 1986 .

[26]  C. L. Hull,et al.  Mechanisms of Adaptive Behavior: Clark L. Hull's Theoretical Papers, with Commentary , 1986 .

[27]  K. Maccorquodale,et al.  Analysis and Integration of Behavioral Units , 1986 .

[28]  A. Dickinson,et al.  Motivational Control of Instrumental Performance: The Role of Prior Experience of The Reinforcer , 1988 .

[29]  A. Catania,et al.  The Selection of Behavior. The Operant Behaviorism of BF Skinner: Comments and Consequences , 1988 .

[30]  A. Amsel,et al.  Behaviorism, Neobehaviorism, and Cognitivism in Learning Theory: Historical and Contemporary Perspectives , 1988 .

[31]  R. Rescorla,et al.  Associations between the discriminative stimulus and the reinforcer in instrumental learning. , 1988 .

[32]  R. Rescorla,et al.  Associations with anticipated and obtained outcomes in instrumental learning , 1989 .

[33]  E. Kehoe.  Connectionist models of conditioning: A tutorial. , 1989, Journal of the experimental analysis of behavior.

[34]  David C. Palmer,et al.  THE INTERPRETATION OF COMPLEX HUMAN BEHAVIOR: SOME REACTIONS TO PARALLEL DISTRIBUTED PROCESSING, EDITED BY J. L. McCLELLAND, D. E. RUMELHART, AND THE PDP RESEARCH GROUP1 , 1989 .

[35]  Robert A. Rescorla,et al.  Effect of reinforcer devaluation on discriminative control of instrumental behavior. , 1990, Journal of experimental psychology. Animal behavior processes.

[36]  M J Watkins,et al.  Mediationism and the Obfuscation of Memory , 1990, The American psychologist.

[37]  R. Rescorla,et al.  Evidence for the hierarchical structure of instrumental learning , 1990 .

[38]  H L Roediger,et al.  Implicit memory. Retention without remembering. , 1990, The American psychologist.

[39]  P. Hineline.  A self-interpretive behavior analysis. , 1992, The American psychologist.

[40]  J. Moore.  On private events and theoretical terms , 1992 .

[41]  J. Donahoe,et al.  Essentialism and selectionism in cognitive science and behavior analysis. , 1992, The American psychologist.

[42]  B. Balleine.  Instrumental performance following a shift in primary motivation depends on incentive learning. , 1992, Journal of experimental psychology. Animal behavior processes.

[43]  D. L. Hintzman.  Twenty-five years of learning and memory: was the cognitive revolution a mistake? , 1993 .

[44]  David C. Palmer,et al.  Learning and Complex Behavior , 1993 .

[45]  L. Stein,et al.  A cellular analogue of operant conditioning. , 1993, Journal of the experimental analysis of behavior.

[46]  E. Kandel,et al.  Effects of cAMP simulate a late stage of LTP in hippocampal CA1 neurons. , 1993, Science.

[47]  J W Donahoe,et al.  A selectionist approach to reinforcement. , 1993, Journal of the experimental analysis of behavior.

[48]  J. Michael.  Establishing Operations , 1993, The Behavior analyst.

[49]  R. Colwill.  An Associative Analysis of Instrumental Learning , 1993 .

[50]  R. Colwill.  Associative Representations of Instrumental Contingencies , 1994 .

[51]  J. Fuster.  Memory in the cerebral cortex , 1994 .

[52]  James C. Houk,et al.  Elements of the Intrinsic Organization and Information Processing in the Neostriatum , 1994 .

[53]  Takashi Yamamoto,et al.  Neural substrates for conditioned taste aversion in the rat , 1994, Behavioural Brain Research.

[54]  B. Williams,et al.  Stimulus devaluation and extinction of chain schedule performance , 1995 .

[55]  J. Wickens,et al.  Cellular models of reinforcement. , 1995 .

[56]  José E. Burgos,et al.  Computational explorations of the evolution of artificial neural networks in Pavlovian environments , 1996 .

[57]  J W Donahoe,et al.  On the Relation Between Behavior Analysis and Biology , 1996, The Behavior analyst.

[58]  J. Fuster.  Memory in the cerebral cortex : an empirical approach to neural networks in the human and nonhuman primate , 1996 .

[59]  E. Bielavská,et al.  Ipsilateral connections between the gustatory cortex, amygdala and parabrachial nucleus are necessary for acquisition and retrieval of conditioned taste aversion in rats , 1996, Behavioural Brain Research.

[60]  MF Bear.  Progress in understanding NMDA-receptor-dependent synaptic plasticity in the visual cortex , 1996, Journal of Physiology-Paris.

[61]  R. Rescorla.  Spontaneous recovery after training with multiple outcomes , 1996 .

[62]  Stephen Maren.  Synaptic transmission and plasticity in the amygdala. An emerging physiology of fear conditioning circuits. , 1996, Molecular neurobiology.

[63]  J. Donoghue,et al.  Long-term potentiation and long-term depression of horizontal connections in rat motor cortex. , 1996, Acta neurobiologiae experimentalis.

[64]  P. Holland,et al.  Neurotoxic Lesions of Basolateral, But Not Central, Amygdala Interfere with Pavlovian Second-Order Conditioning and Reinforcer Devaluation Effects , 1996, The Journal of Neuroscience.

[65]  D. Smith,et al.  Responses of gustatory cells in the nucleus of the solitary tract of the hamster after NaCl or amiloride adaptation. , 1996, Journal of neurophysiology.

[66]  R. Rescorla.  Preservation of Pavlovian Associations through Extinction , 1996 .

[67]  E. Murray,et al.  Excitotoxic Lesions of the Amygdala Fail to Produce Impairment in Visual Learning for Auditory Secondary Reinforcement But Interfere with Reinforcer Devaluation Effects in Rhesus Monkeys , 1997, The Journal of Neuroscience.

[68]  A. Delamater.  Selective reinstatement of stimulus-outcome associations , 1997 .

[69]  U. Frey,et al.  Chapter 6 - Cellular Mechanisms Of Long-Term Potentiation: Late Maintenance , 1997 .

[70]  J. Donahoe.  CHAPTER 18. SELECTION NETWORKS: SIMULATION OF PLASTICITY THROUGH REINFORCEMENT LEARNING , 1997 .

[71]  B. Williams.  What is learned? Revisiting an old issue. , 1997, Journal of the experimental analysis of behavior.

[72]  E T Rolls,et al.  Taste and olfactory processing in the brain and its relation to the control of eating. , 1997, Critical reviews in neurobiology.

[73]  P. Holland,et al.  The Role of an Amygdalo-Nigrostriatal Pathway in Associative Learning , 1997, The Journal of Neuroscience.

[74]  P. Holland.  Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavior theory , 1997 .

[75]  R. Rescorla.  Spontaneous recovery of instrumental discriminative responding , 1997 .

[76]  S. M. Fischer,et al.  ATTENTION AS AN ESTABLISHING OPERATION AND AS REINFORCEMENT DURING FUNCTIONAL ANALYSES , 1997 .

[77]  Jay E. Moore,et al.  The effects of terminal-link stimulus arrangements on preference in concurrent chains , 1997 .

[78]  A. Dickinson.  Bolles's psychological syllogism. , 1997 .

[79]  B. Roche,et al.  The Behavior of Organisms? , 1997 .

[80]  J. Burgos.  Chapter 4 – Evolving Artificial Neural Networks in Pavlovian Environments , 1997 .

[81]  Infants’ Feats of Inference: A Commentary on Bower and Watson , 1997, The Behavior analyst.

[82]  J. Donahoe,et al.  The S-R issue: its status in behavior analysis and in Donahoe and Palmer's learning and complex behavior. , 1997, Journal of the experimental analysis of behavior.

[83]  W Hutchison,et al.  We also need complete behavioral models. , 1997, Journal of the experimental analysis of behavior.

[84]  M. Fanselow,et al.  Distinct Regions of the Periaqueductal Gray Are Involved in the Acquisition and Expression of Defensive Responses , 1998, The Journal of Neuroscience.

[85]  G. Schoenbaum,et al.  Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning , 1998, Nature Neuroscience.

[86]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[87]  G. Aston-Jones,et al.  Origin of noradrenergic afferents to the shell subregion of the nucleus accumbens: anterograde and retrograde tract-tracing studies in the rat , 1998, Brain Research.

[88]  Masataka Watanabe,et al.  Cognitive and Motivational Operations in Primate Prefrontal Neurons , 1998, Reviews in the neurosciences.

[89]  J. Michael.  The Current Status and Future Directions of the Analysis of Verbal Behavior: Comments on the Comments , 1998, The Analysis of verbal behavior.

[90]  J. Donahoe.  Edward L. Thorndike: the selectionist connectionist. , 1999, Journal of the experimental analysis of behavior.

[91]  G. Schoenbaum,et al.  Orbitofrontal Cortex and Representation of Incentive Value in Associative Learning , 1999, The Journal of Neuroscience.

[92]  J W Donahoe,et al.  Timing without a timer. , 1999, Journal of the experimental analysis of behavior.

[93]  J. Staddon,et al.  The dynamics of operant conditioning. , 1999, Psychological review.

[94]  Value transmission in discrimination learning involving stimulus chains. , 1999, Journal of the experimental analysis of behavior.