Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex.

Lesion and functional imaging studies have shown that the ventromedial prefrontal cortex is critically involved in the avoidance of risky choices. However, detailed descriptions of the mechanisms that underlie the establishment of such behaviors remain elusive, due in part to the spatial and temporal limitations of available research techniques. We investigated this issue by recording directly from prefrontal depth electrodes in a rare neurosurgical patient while he performed the Iowa Gambling Task, and we concurrently measured behavioral, autonomic, and electrophysiological responses. We found a robust alpha-band component of event-related potentials that reflected the mismatch between expected outcomes and actual outcomes in the task, correlating closely with the reward-related error obtained from a reinforcement learning model of the patient's choice behavior. The finding implicates this brain region in the acquisition of choice bias by means of a continuous updating of expectations about reward and punishment.

[1]  Clay B. Holroyd,et al.  ERP correlates of feedback and reward processing in the presence and absence of response choice. , 2005, Cerebral cortex.

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Dennis J. L. G. Schutter,et al.  Anterior asymmetrical alpha activity predicts Iowa gambling performance: distinctly but reversed , 2004, Neuropsychologia.

[4]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[5]  D. Tucker,et al.  Frontal midline theta and the error-related negativity: neurophysiological mechanisms of action regulation , 2004, Clinical Neurophysiology.

[6]  M. Herrmann,et al.  Source localization (LORETA) of the error-related-negativity (ERN/Ne) and positivity (Pe). , 2004, Brain research. Cognitive brain research.

[7]  Clay B. Holroyd,et al.  Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback. , 2004, Cerebral cortex.

[8]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[9]  M. Roesch,et al.  Neuronal Activity Related to Reward Value and Motivation in Primate Frontal Cortex , 2004, Science.

[10]  Jeff T. Larsen,et al.  Context dependence of the event-related brain potential associated with reward and punishment. , 2004, Psychophysiology.

[11]  Jonathan D. Cohen,et al.  Anterior Cingulate Conflict Monitoring and Adjustments in Control , 2004, Science.

[12]  Joshua W. Brown,et al.  Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding , 2003, Science.

[13]  Clay B. Holroyd,et al.  Implementation of error-processing in the human anterior cingulate cortex: a source analysis of the magnetic equivalent of the error-related negativity , 2003, Biological Psychology.

[14]  M. Mesulam,et al.  Dissociation of Neural Representation of Intensity and Affective Valuation in Human Gustation , 2003, Neuron.

[15]  J. Price,et al.  Architectonic subdivision of the human orbital and medial prefrontal cortex , 2003, The Journal of comparative neurology.

[16]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[17]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[18]  Bruno A. Olshausen,et al.  Book Review , 2003, Journal of Cognitive Neuroscience.

[19]  Hanna Damasio,et al.  Decision-making and addiction (part I): impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences , 2002, Neuropsychologia.

[20]  J. O'Doherty,et al.  Appetitive and Aversive Olfactory Learning in Humans Studied Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[21]  J. Lisman,et al.  Oscillations in the alpha band (9-12 Hz) increase with memory load during retention in a short-term memory task. , 2002, Cerebral cortex.

[22]  Adrian R. Willoughby,et al.  The Medial Frontal Cortex and the Rapid Processing of Monetary Gains and Losses , 2002, Science.

[23]  J. M. Anderson,et al.  Responses of human frontal cortex to surprising events are predicted by formal associative learning theory , 2001, Nature Neuroscience.

[24]  D. Kahneman,et al.  Functional Imaging of Neural Responses to Expectancy and Experience of Monetary Gains and Losses tasks with monetary payoffs , 2001 .

[25]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[26]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[27]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[28]  W. Schultz,et al.  Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. , 2000, Journal of neurophysiology.

[29]  K. Hikosaka,et al.  Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. , 2000, Cerebral cortex.

[30]  J. Hollerman,et al.  Reward processing in primate orbitofrontal cortex and basal ganglia. , 2000, Cerebral cortex.

[31]  Jonathan D. Cohen,et al.  Conflict monitoring versus selection-for-action in anterior cingulate cortex , 1999, Nature.

[32]  M. Posner The Brain and Emotion , 1999, Nature Medicine.

[33]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[34]  J. Hollerman,et al.  Reward prediction in primate basal ganglia and frontal cortex , 1998, Neuropharmacology.

[35]  H. Damasio,et al.  Dissociation Of Working Memory from Decision Making within the Human Prefrontal Cortex , 1998, The Journal of Neuroscience.

[36]  J. Fuster The Prefrontal Cortex , 1997 .

[37]  A. Damasio,et al.  Deciding Advantageously Before Knowing the Advantageous Strategy , 1997, Science.

[38]  M. Hyyppä Descartes' error. Emotion, reason and the human brain , 1996 .

[39]  M. Howard,et al.  A hybrid clinical-research depth electrode for acute and chronic in vivo microelectrode recording of human brain neurons. Technical note. , 1996, Journal of neurosurgery.

[40]  J. Kagel,et al.  Handbook of Experimental Economics , 1997 .

[41]  G. Schoenbaum,et al.  Information coding in the rodent prefrontal cortex. II. Ensemble activity in orbitofrontal cortex. , 1995, Journal of neurophysiology.

[42]  Sean A. Spence,et al.  Descartes' Error: Emotion, Reason and the Human Brain , 1995 .

[43]  A. Damasio,et al.  Insensitivity to future consequences following damage to human prefrontal cortex , 1994, Cognition.

[44]  A. Damasio Descartes’ Error. Emotion, Reason and the Human Brain. New York (Grosset/Putnam) 1994. , 1994 .

[45]  Robert L. Heilbronner,et al.  The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe, Second Edition , 1989 .

[46]  H. Spinnler The prefrontal cortex, Anatomy, physiology, and neuropsychology of the frontal lobe, J.M. Fuster. Raven Press, New York (1980), IX-222 pages , 1981 .

[47]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[48]  M. Mishkin,et al.  Limbic lesions and the problem of stimulus--reinforcement associations. , 1972, Experimental neurology.

[49]  Burton S. Rosner,et al.  Neuropharmacology , 1958, Nature.