The neural code of reward anticipation in human orbitofrontal cortex

An optimal choice among alternative behavioral options requires precise anticipatory representations of their possible outcomes. A fundamental question is how such anticipated outcomes are represented in the brain. Reward coding at the level of single cells in the orbitofrontal cortex (OFC) follows a more heterogeneous coding scheme than suggested by studies using functional MRI (fMRI) in humans. Using a combination of multivariate pattern classification and fMRI we show that the reward value of sensory cues can be decoded from distributed fMRI patterns in the OFC. This distributed representation is compatible with previous reports from animal electrophysiology that show that reward is encoded by different neural populations with opposing coding schemes. Importantly, the fMRI patterns representing specific values during anticipation are similar to those that emerge during the receipt of reward. Furthermore, we show that the degree of this coding similarity is related to subjects’ ability to use value information to guide behavior. These findings narrow the gap between reward coding in humans and animals and corroborate the notion that value representations in OFC are independent of whether reward is anticipated or actually received.

[1]  G. Schoenbaum,et al.  Information coding in the rodent prefrontal cortex. II. Ensemble activity in orbitofrontal cortex. , 1995, Journal of neurophysiology.

[2]  E. Rolls,et al.  Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. , 1996, Journal of neurophysiology.

[3]  A. Damasio The somatic marker hypothesis and the possible functions of the prefrontal cortex. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[4]  J. B. Levitt,et al.  Patterns of intrinsic and associational circuitry in monkey prefrontal cortex , 1996, The Journal of comparative neurology.

[5]  E. Rolls,et al.  Orbitofrontal cortex neurons: role in olfactory and visual association learning. , 1996, Journal of neurophysiology.

[6]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[7]  A. Damasio,et al.  Deciding Advantageously Before Knowing the Advantageous Strategy , 1997, Science.

[8]  G. Schoenbaum,et al.  Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning , 1998, Nature Neuroscience.

[9]  D. Perrett,et al.  The `Ideal Homunculus': decoding neural population signals , 1998, Trends in Neurosciences.

[10]  Hanna Damasio,et al.  Impairment of social and moral behavior related to early damage in human prefrontal cortex , 1999, Nature Neuroscience.

[11]  G. Schoenbaum,et al.  Orbitofrontal Cortex and Representation of Incentive Value in Associative Learning , 1999, The Journal of Neuroscience.

[12]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[13]  Karl J. Friston,et al.  Stochastic Designs in Event-Related fMRI , 1999, NeuroImage.

[14]  G. Schoenbaum,et al.  Neural Encoding in Orbitofrontal Cortex and Basolateral Amygdala during Olfactory Discrimination Learning , 1999, The Journal of Neuroscience.

[15]  E. Rolls The orbitofrontal cortex and reward. , 2000, Cerebral cortex.

[16]  D. Kahneman,et al.  Functional Imaging of Neural Responses to Expectancy and Experience of Monetary Gains and Losses tasks with monetary payoffs , 2001 .

[17]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[18]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[19]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[20]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[21]  Brian Knutson,et al.  A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI , 2003, NeuroImage.

[22]  J. O'Doherty,et al.  Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex , 2003, Science.

[23]  E. Miller,et al.  Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task , 2003, The European journal of neuroscience.

[24]  Geoffrey Schoenbaum,et al.  Different Roles for Orbitofrontal Cortex and Basolateral Amygdala in a Reinforcer Devaluation Task , 2003, The Journal of Neuroscience.

[25]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[26]  M. Roesch,et al.  Neuronal Activity Related to Reward Value and Motivation in Primate Frontal Cortex , 2004, Science.

[27]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[28]  M. Kringelbach The human orbitofrontal cortex: linking reward to hedonic experience , 2005, Nature Reviews Neuroscience.

[29]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[30]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[31]  J. O'Doherty,et al.  Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain , 2006, PLoS biology.

[32]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[33]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[35]  J. Wallis Orbitofrontal cortex and its contribution to decision-making. , 2007, Annual review of neuroscience.

[36]  R. Passingham,et al.  Reading Hidden Intentions in the Human Brain , 2007, Current Biology.

[37]  J. O'Doherty,et al.  Lights, Camembert, Action! The Role of Human Orbitofrontal Cortex in Encoding Stimuli, Rewards, and Choices , 2007, Annals of the New York Academy of Sciences.

[38]  G. Schoenbaum,et al.  Reconciling the Roles of Orbitofrontal Cortex in Reversal Learning and the Encoding of Outcome Expectancies , 2007, Annals of the New York Academy of Sciences.

[39]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.

[40]  J. O'Doherty,et al.  What We Know and Do Not Know about the Functions of the Orbitofrontal Cortex after 20 Years of Cross-Species Studies , 2007, The Journal of Neuroscience.

[41]  J. O'Doherty,et al.  Decoding the neural substrates of reward-related decision making with functional MRI , 2007, Proceedings of the National Academy of Sciences.

[42]  C. Pennartz,et al.  Population Coding of Reward Magnitude in the Orbitofrontal Cortex of the Rat , 2008, The Journal of Neuroscience.

[43]  J. O'Doherty,et al.  Marketing actions can modulate neural representations of experienced pleasantness , 2008, Proceedings of the National Academy of Sciences.

[44]  M. Brass,et al.  Unconscious determinants of free decisions in the human brain , 2008, Nature Neuroscience.

[45]  R. Quiroga,et al.  Extracting information from neuronal populations: information theory and decoding approaches , 2009, Nature Reviews Neuroscience.

[46]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[47]  S. Kennerley,et al.  Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables , 2009, The European journal of neuroscience.

[48]  Jonathan D. Wallis,et al.  Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables , 2009, Journal of Cognitive Neuroscience.

[49]  Steven W Kennerley,et al.  Encoding of Gustatory Working Memory by Orbitofrontal Neurons , 2009, The Journal of Neuroscience.

[50]  C. Pennartz,et al.  Single-Cell and Population Coding of Expected Reward Probability in the Orbitofrontal Cortex of the Rat , 2009, The Journal of Neuroscience.

[51]  C. Daniel Salzman,et al.  The Convergence of Information about Rewarding and Aversive Stimuli in Single Neurons , 2009, The Journal of Neuroscience.

[52]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .