Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning

Instrumental conditioning studies how animals and humans choose actions appropriate to the affective structure of an environment. According to recent reinforcement learning models, two distinct components are involved: a “critic,” which learns to predict future reward, and an “actor,” which maintains information about the rewarding outcomes of actions to enable better ones to be chosen more frequently. We scanned human participants with functional magnetic resonance imaging while they engaged in instrumental conditioning. Our results suggest partly dissociable contributions of the ventral and dorsal striatum, with the former corresponding to the critic and the latter corresponding to the actor.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  Burton S. Rosner,et al.  Neuropharmacology , 1956, Nature.

[3]  R J Herrnstein,et al.  Formal properties of the matching law. , 1974, Journal of the experimental analysis of behavior.

[4]  A. Dickinson.  Conditioning and associative learning. , 1981, British medical bulletin.

[5]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[6]  W. Schultz,et al.  Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. , 1990, Journal of neurophysiology.

[7]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[8]  C. Gerfen.  The neostriatal mosaic: multiple levels of compartmental organization , 1992, Trends in Neurosciences.

[9]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[10]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[11]  A. Graybiel.  Building action repertoires: memory and learning functions of the basal ganglia , 1995, Current Opinion in Neurobiology.

[12]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[13]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[15]  N. White.  Mnemonic functions of the basal ganglia , 1997, Current Opinion in Neurobiology.

[16]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[17]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[18]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[19]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[20]  Joshua W. Brown,et al.  How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues , 1999, The Journal of Neuroscience.

[21]  Ivan Toni,et al.  Prefrontal-basal ganglia pathways are involved in the learning of arbitrary visuomotor associations: a PET study , 1999, Experimental Brain Research.

[22]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[23]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[24]  B. Knowlton,et al.  Learning and memory functions of the Basal Ganglia. , 2002, Annual review of neuroscience.

[25]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[26]  Eytan Ruppin,et al.  Actor-critic models of the basal ganglia: new anatomical and computational perspectives , 2002, Neural Networks.

[27]  B. Everitt,et al.  Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex , 2002, Neuroscience & Biobehavioral Reviews.

[28]  P. Montague,et al.  Activity in human ventral striatum locked to errors of reward prediction , 2002, Nature Neuroscience.

[29]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[30]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[31]  M. Delgado,et al.  Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations , 2003, Cognitive, affective & behavioral neuroscience.

[32]  S. Killcross,et al.  Coordination of actions and habits in the medial prefrontal cortex of rats. , 2003, Cerebral cortex.

[33]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.