Behavior analysis and revaluation.

Revaluation refers to phenomena in which the strength of an operant is altered by reinforcer-related manipulations that take place outside the conditioning situation in which the operant was selected. As an example, if lever pressing is acquired using food as a reinforcer and food is later paired with an aversive stimulus, the frequency of lever pressing decreases when subsequently tested. Associationist psychology infers from such findings that conditioning produces a response-outcome (i.e., reinforcer) association and that the operant decreased in strength because pairing the reinforcer with the aversive stimulus changed the value of the outcome. Here, we present an approach to the interpretation of these and related findings that employs neural network simulations grounded in the experimental analysis of behavior and neuroscience. In so doing, we address some general issues regarding the relations among behavior analysis, neuroscience, and associationism.

[1]  C. L. Hull Knowledge and purpose as habit mechanisms. , 1930 .

[2]  C. L. Hull Goal attraction and directing ideas conceived as habit phenomena. , 1931 .

[3]  B. Skinner Two Types of Conditioned Reflex and a Pseudo Type , 1935 .

[4]  J. Konorski,et al.  On Two Types of Conditioned Reflex , 1937 .

[5]  B. Skinner,et al.  Are theories of learning necessary? , 2013, Psychological review.

[6]  B. Skinner,et al.  Science and human behavior , 1953 .

[7]  C. Hempel,et al.  Aspects of Scientific Explanation and Other Essays in the Philosophy of Science. , 1966 .

[8]  J. E. Mazur,et al.  Learning and Behavior , 1966 .

[9]  B. Skinner What is the experimental analysis of behavior? , 1966, Journal of the experimental analysis of behavior.

[10]  R. Rescorla,et al.  Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. , 1967, Psychological review.

[11]  R. Kovner,et al.  Cue properties vs palatability of flavors in avoidance learning , 1970 .

[12]  T. Bliss,et al.  Long‐lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path , 1973, The Journal of physiology.

[13]  R. Rescorla,et al.  Reinstatement of fear to an extinguished conditioned stimulus. , 1975, Journal of experimental psychology. Animal behavior processes.

[14]  R. Rescorla,et al.  The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. , 1975, Journal of experimental psychology. Animal behavior processes.

[15]  John R. Anderson Arguments concerning representations for mental imagery. , 1978 .

[16]  E. Fantino,et al.  Effects on choice of reinforcement delay and conditioned reinforcement. , 1978, Journal of the experimental analysis of behavior.

[17]  J. Overmier,et al.  Pavlovian Conditioning and the Mediation of behavior1 , 1979 .

[18]  W. K. Honig,et al.  Cognitive Processes in Animal Behavior , 1979 .

[19]  P. Chance Learning and Behavior , 1979 .

[20]  J. Fuster Prefrontal Cortex , 2018 .

[21]  J. Michael Distinguishing between discriminative and motivational functions of stimuli. , 1982, Journal of the experimental analysis of behavior.

[22]  David M. Sargent,et al.  The order of presentation of conditioned stimuli during extinction , 1983 .

[23]  R. Rescorla,et al.  Postconditioning devaluation of a reinforcer affects instrumental responding. , 1985 .

[24]  B. Skinner Is it behaviorism? , 1986, Behavioral and Brain Sciences.

[25]  B. Williams Identifying Behaviorism’s Prototype: A Review of Behaviorism: A Conceptual Reconstruction by G. E. Zuriff , 1986 .

[26]  C. L. Hull,et al.  Mechanisms of Adaptive Behavior: Clark L. Hull's Theoretical Papers, with Commentary , 1986 .

[27]  K. Maccorquodale,et al.  Analysis and Integration of Behavioral Units , 1986 .

[28]  A. Dickinson,et al.  Motivational Control of Instrumental Performance: The Role of Prior Experience of The Reinforcer , 1988 .

[29]  A. Catania,et al.  The Selection of Behavior. The Operant Behaviorism of BF Skinner: Comments and Consequences , 1988 .

[30]  A. Amsel,et al.  Behaviorism, Neobehaviorism, and Cognitivism in Learning Theory: Historical and Contemporary Perspectives , 1988 .

[31]  R. Rescorla,et al.  Associations between the discriminative stimulus and the reinforcer in instrumental learning. , 1988 .

[32]  R. Rescorla,et al.  Associations with anticipated and obtained outcomes in instrumental learning , 1989 .

[33]  E. Kehoe Connectionist models of conditioning: A tutorial. , 1989, Journal of the experimental analysis of behavior.

[34]  David C. Palmer,et al.  THE INTERPRETATION OF COMPLEX HUMAN BEHAVIOR: SOME REACTIONS TO PARALLEL DISTRIBUTED PROCESSING, EDITED BY J. L. McCLELLAND, D. E. RUMELHART, AND THE PDP RESEARCH GROUP1 , 1989 .

[35]  Robert A. Rescorla,et al.  Effect of reinforcer devaluation on discriminative control of instrumental behavior. , 1990, Journal of experimental psychology. Animal behavior processes.

[36]  M J Watkins,et al.  Mediationism and the Obfuscation of Memory , 1990, The American psychologist.

[37]  R. Rescorla,et al.  Evidence for the hierarchical structure of instrumental learning , 1990 .

[38]  H L Roediger,et al.  Implicit memory. Retention without remembering. , 1990, The American psychologist.

[39]  P. Hineline A self-interpretive behavior analysis. , 1992, The American psychologist.

[40]  J. Moore On private events and theoretical terms , 1992 .

[41]  J. Donahoe,et al.  Essentialism and selectionism in cognitive science and behavior analysis. , 1992, The American psychologist.

[42]  B. Balleine Instrumental performance following a shift in primary motivation depends on incentive learning. , 1992, Journal of experimental psychology. Animal behavior processes.

[43]  D. L. Hintzman Twenty-five years of learning and memory: was the cognitive revolution a mistake? , 1993 .

[44]  David C. Palmer,et al.  Learning and Complex Behavior , 1993 .

[45]  L. Stein,et al.  A cellular analogue of operant conditioning. , 1993, Journal of the experimental analysis of behavior.

[46]  E. Kandel,et al.  Effects of cAMP simulate a late stage of LTP in hippocampal CA1 neurons. , 1993, Science.

[47]  J W Donahoe,et al.  A selectionist approach to reinforcement. , 1993, Journal of the experimental analysis of behavior.

[48]  J. Michael Establishing Operations , 1993, The Behavior analyst.

[49]  R. Colwill An Associative Analysis of Instrumental Learning , 1993 .

[50]  R. Colwill Associative Representations of Instrumental Contingencies , 1994 .

[51]  J. Fuster Memory in the cerebral cortex , 1994 .

[52]  James C. Houk,et al.  Elements of the Intrinsic Organization and Information Processing in the Neostriatum , 1994 .

[53]  Takashi Yamamoto,et al.  Neural substrates for conditioned taste aversion in the rat , 1994, Behavioural Brain Research.

[54]  B. Williams,et al.  Stimulus devaluation and extinction of chain schedule performance , 1995 .

[55]  J. Wickens,et al.  Cellular models of reinforcement. , 1995 .

[56]  José E. Burgos,et al.  Computational explorations of the evolution of artificial neural networks in Pavlovian environments , 1996 .

[57]  J W Donahoe,et al.  On the Relation Between Behavior Analysis and Biology , 1996, The Behavior analyst.

[58]  J. Fuster Memory in the cerebral cortex : an empirical approach to neural networks in the human and nonhuman primate , 1996 .

[59]  E. Bielavská,et al.  Ipsilateral connections between the gustatory cortex, amygdala and parabrachial nucleus are necessary for acquisition and retrieval of conditioned taste aversion in rats , 1996, Behavioural Brain Research.

[60]  MF Bear Progress in understanding NMDA-receptor-dependent synaptic plasticity in the visual cortex , 1996, Journal of Physiology - Paris.

[61]  R. Rescorla Spontaneous recovery after training with multiple outcomes , 1996 .

[62]  Stephen Maren Synaptic transmission and plasticity in the amygdala. An emerging physiology of fear conditioning circuits. , 1996, Molecular neurobiology.

[63]  J. Donoghue,et al.  Long-term potentiation and long-term depression of horizontal connections in rat motor cortex. , 1996, Acta neurobiologiae experimentalis.

[64]  P. Holland,et al.  Neurotoxic Lesions of Basolateral, But Not Central, Amygdala Interfere with Pavlovian Second-Order Conditioning and Reinforcer Devaluation Effects , 1996, The Journal of Neuroscience.

[65]  D. Smith,et al.  Responses of gustatory cells in the nucleus of the solitary tract of the hamster after NaCl or amiloride adaptation. , 1996, Journal of neurophysiology.

[66]  R. Rescorla Preservation of Pavlovian Associations through Extinction , 1996 .

[67]  E. Murray,et al.  Excitotoxic Lesions of the Amygdala Fail to Produce Impairment in Visual Learning for Auditory Secondary Reinforcement But Interfere with Reinforcer Devaluation Effects in Rhesus Monkeys , 1997, The Journal of Neuroscience.

[68]  A. Delamater Selective reinstatement of stimulus-outcome associations , 1997 .

[69]  U. Frey,et al.  Chapter 6 - Cellular Mechanisms Of Long-Term Potentiation: Late Maintenance , 1997 .

[70]  J. Donahoe CHAPTER 18. SELECTION NETWORKS: SIMULATION OF PLASTICITY THROUGH REINFORCEMENT LEARNING , 1997 .

[71]  B. Williams What is learned? Revisiting an old issue. , 1997, Journal of the experimental analysis of behavior.

[72]  E T Rolls,et al.  Taste and olfactory processing in the brain and its relation to the control of eating. , 1997, Critical reviews in neurobiology.

[73]  P. Holland,et al.  The Role of an Amygdalo-Nigrostriatal Pathway in Associative Learning , 1997, The Journal of Neuroscience.

[74]  P. Holland Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavior theory , 1997 .

[75]  R. Rescorla Spontaneous recovery of instrumental discriminative responding , 1997 .

[76]  S. M. Fischer,et al.  ATTENTION AS AN ESTABLISHING OPERATION AND AS REINFORCEMENT DURING FUNCTIONAL ANALYSES , 1997 .

[77]  Jay E. Moore,et al.  The effects of terminal-link stimulus arrangements on preference in concurrent chains , 1997 .

[78]  A. Dickinson Bolles's psychological syllogism. , 1997 .

[79]  B. Roche,et al.  The Behavior of Organisms? , 1997 .

[80]  J. Burgos Chapter 4 – Evolving Artificial Neural Networks in Pavlovian Environments , 1997 .

[81]  Infants’ Feats of Inference: A Commentary on Bower and Watson , 1997, The Behavior analyst.

[82]  J. Donahoe,et al.  The S-R issue: its status in behavior analysis and in Donahoe and Palmer's learning and complex behavior. , 1997, Journal of the experimental analysis of behavior.

[83]  W Hutchison We also need complete behavioral models. , 1997, Journal of the experimental analysis of behavior.

[84]  M. Fanselow,et al.  Distinct Regions of the Periaqueductal Gray Are Involved in the Acquisition and Expression of Defensive Responses , 1998, The Journal of Neuroscience.

[85]  G. Schoenbaum,et al.  Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning , 1998, Nature Neuroscience.

[86]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[87]  G. Aston-Jones,et al.  Origin of noradrenergic afferents to the shell subregion of the nucleus accumbens: anterograde and retrograde tract-tracing studies in the rat , 1998, Brain Research.

[88]  Masataka Watanabe,et al.  Cognitive and Motivational Operations in Primate Prefrontal Neurons , 1998, Reviews in the neurosciences.

[89]  J. Michael The Current Status and Future Directions of the Analysis of Verbal Behavior: Comments on the Comments , 1998, The Analysis of verbal behavior.

[90]  J. Donahoe Edward L. Thorndike: the selectionist connectionist. , 1999, Journal of the experimental analysis of behavior.

[91]  G. Schoenbaum,et al.  Orbitofrontal Cortex and Representation of Incentive Value in Associative Learning , 1999, The Journal of Neuroscience.

[92]  J W Donahoe,et al.  Timing without a timer. , 1999, Journal of the experimental analysis of behavior.

[93]  J. Staddon,et al.  The dynamics of operant conditioning. , 1999, Psychological review.

[94]  Value transmission in discrimination learning involving stimulus chains. , 1999, Journal of the experimental analysis of behavior.