Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala

The discovery that dopamine neurons signal errors in reward prediction has demonstrated that concepts empirically derived from the study of animal behavior can be used to understand the neural implementation of reward learning. Yet the learning theory models linked to phasic dopamine activity treat attention to events such as cues and rewards as static quantities; other models, such as Pearce–Hall, propose that learning might be influenced by variations in processing of these events. A key feature of these accounts is that event processing is modulated by unsigned rather than signed reward prediction errors. Here we tested whether neural activity in rat basolateral amygdala conforms to this pattern by recording single units in a behavioral task in which rewards were unexpectedly delivered or omitted. We report that neural activity at the time of reward is providing an unsigned error signal with characteristics consistent with those postulated by these models. This neural signal increased immediately after a change in reward, and stronger firing was evident whether the value of the reward increased or decreased. Further, as predicted by these models, the change in firing developed over several trials as expectations for reward were repeatedly violated. This neural signal was correlated with faster orienting to predictive cues after changes in reward, and abolition of the signal by inactivation of basolateral amygdala disrupted this change in orienting and retarded learning in response to changes in reward. These results suggest that basolateral amygdala serves a critical function in attention for learning.

[1]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[2]  N. Mackintosh A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[3]  J. Pearce,et al.  Latent inhibition of a CS during CS-US pairings. , 1979, Journal of experimental psychology. Animal behavior processes.

[4]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[5]  J. Pearce,et al.  The strength of the orienting response during Pavlovian conditioning. , 1984, Journal of experimental psychology. Animal behavior processes.

[6]  J. Pearce,et al.  The Influence of Predictive Accuracy on Serial Conditioning in the Rat , 1988 .

[7]  H Nishijo,et al.  Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  J. Pearce,et al.  The orienting response as an index of stimulus associability in rats. , 1988, Journal of experimental psychology. Animal behavior processes.

[9]  J. Pearce,et al.  Restoration of the Orienting Response to a Light by a Change in its Predictive Accuracy , 1992 .

[10]  M. Gallagher,et al.  Amygdala central nucleus lesions disrupt increments, but not decrements, in conditioned stimulus processing. , 1993, Behavioral neuroscience.

[11]  G. Aston-Jones,et al.  Locus coeruleus activity in monkey: Phasic and tonic changes are associated with altered vigilance , 1994, Brain Research Bulletin.

[12]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[13]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[15]  G. Schoenbaum,et al.  Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning , 1998, Nature Neuroscience.

[16]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[17]  P. Holland,et al.  Amygdala circuitry in attentional and representational processes , 1999, Trends in Cognitive Sciences.

[18]  G. Schoenbaum,et al.  Neural Encoding in Orbitofrontal Cortex and Basolateral Amygdala during Olfactory Discrimination Learning , 1999, The Journal of Neuroscience.

[19]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[20]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[21]  Michael Davis,et al.  The amygdala: vigilance and emotion , 2001, Molecular Psychiatry.

[22]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[23]  G. Schoenbaum,et al.  Encoding Predicted Outcome and Acquired Value in Orbitofrontal Cortex during Cue Sampling Depends upon Input from Basolateral Amygdala , 2003, Neuron.

[24]  Geoffrey Schoenbaum,et al.  Different Roles for Orbitofrontal Cortex and Basolateral Amygdala in a Reinforcer Devaluation Task , 2003, The Journal of Neuroscience.

[25]  M. L. Le Pelley The Role of Associative History in Models of Associative Learning: A Selective Review and a Hybrid Model , 2004, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[26]  B. Richmond,et al.  Neuronal Signals in the Monkey Basolateral Amygdala during Reward Schedules , 2005, The Journal of Neuroscience.

[27]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[28]  P. Holland,et al.  Variations in unconditioned stimulus processing in unblocking. , 2005, Journal of experimental psychology. Animal behavior processes.

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[31]  G. Hall,et al.  Modulation of the effective salience of a stimulus by direct and associative activation of its representation. , 2005, Journal of experimental psychology. Animal behavior processes.

[32]  Joseph J. Paton,et al.  The primate amygdala represents the positive and negative value of visual stimuli during learning , 2006, Nature.

[33]  A. Lüthi,et al.  Processing of Temporal Unpredictability in Human and Animal Amygdala , 2007, The Journal of Neuroscience.

[34]  P. Glimcher,et al.  Statistics of midbrain dopamine neuron spike trains in the awake primate. , 2007, Journal of neurophysiology.

[35]  Joseph J. Paton,et al.  Expectation Modulates Neural Responses to Pleasant and Aversive Stimuli in Primate Amygdala , 2007, Neuron.

[36]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[37]  E. Murray The amygdala, reward and emotion , 2007, Trends in Cognitive Sciences.

[38]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[39]  Hatim A. Zariwala,et al.  Neural correlates, computation and behavioural impact of decision confidence , 2008, Nature.

[40]  Simon Hong,et al.  The Globus Pallidus Sends Reward-Related Signals to the Lateral Habenula , 2008, Neuron.

[41]  Kay M. Tye,et al.  Rapid strengthening of thalamo-amygdala synapses mediates cue–reward learning , 2008, Nature.

[42]  Samuel M. McClure,et al.  BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area , 2008, Science.

[43]  M. Nicolelis,et al.  Neuronal Ensemble Bursting in the Basal Forebrain Encodes Salience Irrespective of Valence , 2008, Neuron.

[44]  S. Nicola,et al.  Basolateral Amygdala Neurons Facilitate Reward-Seeking Behavior by Exciting Nucleus Accumbens Neurons , 2008, Neuron.

[45]  Joseph J. Paton,et al.  Moment-to-Moment Tracking of State Value in the Amygdala , 2008, The Journal of Neuroscience.

[46]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[47]  D. Katz,et al.  Distinct Subtypes of Basolateral Amygdala Taste Neurons Reflect Palatability and Reward , 2009, The Journal of Neuroscience.

[48]  Jackson J. Cone,et al.  Amygdala Neural Encoding of the Absence of Reward during Extinction , 2010, The Journal of Neuroscience.