Responses of Amygdala Neurons to Positive Reward-Predicting Stimuli Depend on Background Reward (Contingency) Rather Than Stimulus-Reward Pairing (Contiguity)

Prediction about outcomes constitutes a basic mechanism underlying informed economic decision making. A stimulus constitutes a reward predictor when it provides more information about the reward than the environmental background. Reward prediction can be manipulated in two ways, by varying the reward paired with the stimulus, as done traditionally in neurophysiological studies, and by varying the background reward while holding stimulus-reward pairing constant. Neuronal mechanisms involved in reward prediction should also be sensitive to changes in background reward independently of stimulus-reward pairing. We tested this assumption on a major brain structure involved in reward processing, the central and basolateral amygdala. In a 2 × 2 design, we examined the influence of rewarded and unrewarded backgrounds on neuronal responses to rewarded and unrewarded visual stimuli. Indeed, responses to the unchanged rewarded stimulus depended crucially on background reward in a population of amygdala neurons. Elevating background reward to the level of the rewarded stimulus extinguished these responses, and lowering background reward again reinstated the responses without changes in stimulus-reward pairing. None of these neurons responded specifically to an inhibitory stimulus predicting less reward compared with background (negative contingency). A smaller group of amygdala neurons maintained stimulus responses irrespective of background reward, possibly reflecting stimulus-reward pairing or visual sensory processes without reward prediction. Thus in being sensitive to background reward, the responses of a population of amygdala neurons to phasic stimuli appeared to follow the full criteria for excitatory reward prediction (positive contingency) rather than reflecting simple stimulus-reward pairing (contiguity).

[1]  M. D. Egger,et al.  Secondary reinforcement in rats as a function of information value and reliability of the stimulus. , 1962, Journal of experimental psychology.

[2]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[3]  E. Fischer Conditioned Reflexes , 1942, American journal of physical medicine.

[4]  R. Rescorla Pavlovian conditioning and its proper control procedures. , 1967, Psychological review.

[5]  R. Rescorla Probability of shock in the presence and absence of CS in fear conditioning. , 1968, Journal of comparative and physiological psychology.

[6]  R. Rescorla Conditioned inhibition of fear resulting from negative CS-US contingencies. , 1969, Journal of comparative and physiological psychology.

[7]  A. R. Wagner,et al.  Situational cues and correlation between CS and US as determinants of the conditioned emotional response , 1970 .

[8]  R. Wurtz,et al.  Activity of superior colliculus in behaving monkey. II. Effect of attention on neuronal responses. , 1972, Journal of neurophysiology.

[9]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[10]  A. Baker Conditioned Inhibition Arising from a Between-Sessions Negative Correlation. , 1977 .

[11]  E. Rolls,et al.  Visual responses of neurons in the dorsolateral amygdala of the alert monkey , 1979, Experimental Neurology.

[12]  A. Dickinson Contemporary Animal Learning Theory , 1981 .

[13]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[14]  R. Roth,et al.  Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: An anterograde tract‐tracing study with Phaseolus vulgaris leucoagglutinin , 1989, The Journal of comparative neurology.

[15]  T. Robbins,et al.  The basolateral amygdala-ventral striatal system and conditioned place preference: Further evidence of limbic-striatal interactions underlying reward-related processes , 1991, Neuroscience.

[16]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[17]  M. Bouton,et al.  Effects of Contextual Conditioning and Unconditional Stimulus Presentation on Performance in Appetitive Conditioning , 1993, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[18]  M. Fabre-Thorpe,et al.  Interaction of the Amygdala with the Frontal Lobe in Reward Memory , 1993, The European journal of neuroscience.

[19]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[20]  M. Davis,et al.  Lesions of the central nucleus of the amygdala block conditioned excitation, but not conditioned inhibition of fear as measured with the fear-potentiated startle effect. , 1995, Behavioral neuroscience.

[21]  Hisao Nishijo,et al.  Amygdala role in conditioned associative learning , 1995, Progress in Neurobiology.

[22]  A. Delamater Outcome-selective effects of intertrial reinforcement in a Pavlovian appetitive conditioning paradigm with rats , 1995 .

[23]  M. Fanselow,et al.  Synaptic plasticity in the basolateral amygdala induced by hippocampal formation stimulation in vivo , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  P. Holland,et al.  Neurotoxic Lesions of Basolateral, But Not Central, Amygdala Interfere with Pavlovian Second-Order Conditioning and Reinforcer Devaluation Effects , 1996, The Journal of Neuroscience.

[26]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[27]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[28]  E. Murray,et al.  Excitotoxic Lesions of the Amygdala Fail to Produce Impairment in Visual Learning for Auditory Secondary Reinforcement But Interfere with Reinforcer Devaluation Effects in Rhesus Monkeys , 1997, The Journal of Neuroscience.

[29]  P. Holland,et al.  The Role of an Amygdalo-Nigrostriatal Pathway in Associative Learning , 1997, The Journal of Neuroscience.

[30]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[31]  J. Tanji,et al.  Role for cingulate motor area cells in voluntary movement selection based on reward. , 1998, Science.

[32]  S. Mizumori,et al.  Characteristics of basolateral amygdala neuronal firing on a spatial memory task involving differential reward. , 1998, Behavioral neuroscience.

[33]  A. Toga,et al.  The Rhesus Monkey Brain in Stereotaxic Coordinates , 1999 .

[34]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[35]  S. Wise,et al.  Rule-dependent neuronal activity in the prefrontal cortex , 1999, Experimental Brain Research.

[36]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[37]  M. Shadlen,et al.  Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque , 1999, Nature Neuroscience.

[38]  G. Schoenbaum,et al.  Neural Encoding in Orbitofrontal Cortex and Basolateral Amygdala during Olfactory Discrimination Learning , 1999, The Journal of Neuroscience.

[39]  E. Murray,et al.  Control of Response Selection by Reinforcer Value Requires Interaction of Amygdala and Orbital Prefrontal Cortex , 2000, The Journal of Neuroscience.

[40]  A. Ylinen,et al.  Reciprocal Connections between the Amygdala and the Hippocampal Formation, Perirhinal Cortex, and Postrhinal Cortex in Rat: A Review , 2000, Annals of the New York Academy of Sciences.

[41]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[42]  Barry Setlow,et al.  Disconnection of the basolateral amygdala complex and nucleus accumbens impairs appetitive pavlovian second-order conditioned responses. , 2002, Behavioral neuroscience.

[43]  Trevor W Robbins,et al.  Appetitive Behavior , 2003, Annals of the New York Academy of Sciences.

[44]  R. Carelli,et al.  Basolateral Amygdala Neurons Encode Cocaine Self-Administration and Cocaine-Associated Cues , 2003, The Journal of Neuroscience.

[45]  Wolfram Schultz,et al.  Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. , 2003, Journal of neurophysiology.

[46]  C. Gallistel,et al.  Conditioning from an information processing perspective , 2003, Behavioural Processes.

[47]  W. Schultz,et al.  Role of primate basal ganglia and frontal cortex in the internal generation of movements , 1992, Experimental Brain Research.

[48]  E. Phelps Human emotion and memory: interactions of the amygdala and hippocampal complex , 2004, Current Opinion in Neurobiology.

[49]  R. Andersen,et al.  Cognitive Control Signals for Neural Prosthetics , 2004, Science.

[50]  R. Passingham,et al.  Stereotaxic surgery under X-ray guidance in the rhesus monkey, with special reference to the amygdala , 2004, Experimental Brain Research.

[51]  R. Dolan,et al.  Encoding of emotional memories depends on amygdala and hippocampus and their interactions , 2004, Nature Neuroscience.

[52]  J. Maunsell Neuronal representations of cognitive state: reward or attention? , 2004, Trends in Cognitive Sciences.

[53]  B. Richmond,et al.  Neuronal Signals in the Monkey Basolateral Amygdala during Reward Schedules , 2005, The Journal of Neuroscience.

[54]  Geoffrey Schoenbaum,et al.  Rapid Associative Encoding in Basolateral Amygdala Depends on Connections with Orbitofrontal Cortex , 2005, Neuron.

[55]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[56]  M. Shadlen,et al.  A representation of the hazard rate of elapsed time in macaque area LIP , 2005, Nature Neuroscience.

[57]  R. Dolan,et al.  Task and Content Modulate Amygdala-Hippocampal Connectivity in Emotional Retrieval , 2006, Neuron.

[58]  C. Gallistel,et al.  Pavlovian contingencies and temporal information. , 2006, Journal of experimental psychology. Animal behavior processes.

[59]  Joseph J. Paton,et al.  The primate amygdala represents the positive and negative value of visual stimuli during learning , 2006, Nature.

[60]  Kay M. Tye,et al.  Amygdala Neurons Differentially Encode Motivation and Reinforcement , 2007, The Journal of Neuroscience.

[61]  Joseph J. Paton,et al.  Expectation Modulates Neural Responses to Pleasant and Aversive Stimuli in Primate Amygdala , 2007, Neuron.

[62]  B. Balleine,et al.  Differential Involvement of the Basolateral Amygdala and Mediodorsal Thalamus in Instrumental Action Selection , 2008, The Journal of Neuroscience.

[63]  Joseph J. Paton,et al.  Moment-to-Moment Tracking of State Value in the Amygdala , 2008, The Journal of Neuroscience.

[64]  C. Gallistel,et al.  Temporal maps and informativeness in associative learning , 2009, Trends in Neurosciences.

[65]  E. Miller,et al.  Learning Substrates in the Primate Prefrontal Cortex and Striatum: Sustained Activity Related to Successful Actions , 2009, Neuron.

[66]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.