Human Insula Activation Reflects Risk Prediction Errors As Well As Risk

Understanding how organisms deal with probabilistic stimulus-reward associations has been advanced by a convergence between reinforcement learning models and primate physiology, which demonstrated that the brain encodes a reward prediction error signal. However, organisms must also predict the level of risk associated with reward forecasts, monitor the errors in those risk predictions, and update these in light of new information. Risk prediction serves a dual purpose: (1) to guide choice in risk-sensitive organisms and (2) to modulate learning of uncertain rewards. To date, it is not known whether or how the brain accomplishes risk prediction. Using functional imaging during a simple gambling task in which we constantly changed risk, we show that an early-onset activation in the human insula correlates significantly with risk prediction error and that its time course is consistent with a role in rapid updating. Additionally, we show that activation previously associated with general uncertainty emerges with a delay consistent with a role in risk prediction. The activations correlating with risk prediction and risk prediction errors are the analogy for risk of activations correlating with reward prediction and reward prediction errors for reward expectation. As such, our findings indicate that our understanding of the neural basis of reward anticipation under uncertainty needs to be expanded to include risk prediction.

[1]  F. Black,et al.  The Pricing of Options and Corporate Liabilities , 1973, Journal of Political Economy.

[2]  R. Engle Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[3]  T. Caraco Aspects of risk-aversion in foraging white-crowned sparrows , 1982, Animal Behaviour.

[4]  M. Ma,et al.  FOUNDATIONS OF PORTFOLIO THEORY , 1990 .

[5]  L A Real,et al.  Animal choice behavior and the evolution of cognitive architecture , 1991, Science.

[6]  Reid G. Simmons,et al.  Risk-Sensitive Planning with Probabilistic Decision Graphs , 1994, KR.

[7]  Matthias Heger,et al.  Consideration of Risk in Reinforcement Learning , 1994, ICML.

[8]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[9]  K. Nader,et al.  Neurobiological constraints on behavioral models of motivation. , 1997, Annual review of psychology.

[10]  M. Mesulam,et al.  From sensation to cognition. , 1998, Brain : a journal of neurology.

[11]  Y. Dudai,et al.  The Role of Identified Neurotransmitter Systems in the Response of Insular Cortex to Unfamiliar Taste: Activation of ERK1–2 and Formation of a Memory Trace , 2000, The Journal of Neuroscience.

[12]  Karl J. Friston,et al.  Dissociable Neural Responses in Human Reward Systems , 2000, The Journal of Neuroscience.

[13]  H. Critchley,et al.  Neural Activity in the Human Brain Relating to Uncertainty and Arousal during Anticipation , 2001, Neuron.

[14]  Peter Dayan,et al.  Expected and Unexpected Uncertainty: ACh and NE in the Neocortex , 2002, NIPS.

[15]  R. Engle New Frontiers for Arch Models , 2002 .

[16]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[17]  Monique Ernst,et al.  Decision-making in a Risk-taking Task: A PET Study , 2002, Neuropsychopharmacology.

[18]  A. Craig How do you feel? Interoception: the sense of the physiological condition of the body , 2002, Nature Reviews Neuroscience.

[19]  Corianne Rogalsky,et al.  Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism , 2003, NeuroImage.

[20]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[21]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[22]  Matthew D. Lieberman,et al.  Does Rejection Hurt? An fMRI Study of Social Exclusion , 2003, Science.

[23]  Ralph Neuneier,et al.  Risk-Sensitive Reinforcement Learning , 1998, Machine Learning.

[24]  Jonathan D. Cohen,et al.  Computational roles for dopamine in behavioural control , 2004, Nature.

[25]  W. Schultz Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology , 2004, Current Opinion in Neurobiology.

[26]  E. Weber,et al.  Predicting Risk-Sensitivity in Humans and Lower Animals: Risk as Variance or Coefficient of Variation , 2004, Psychological review.

[27]  C. Barnard,et al.  Risk-sensitive foraging in common shrews (Sorex araneus L.) , 2004, Behavioral Ecology and Sociobiology.

[28]  J. O'Doherty,et al.  Empathy for Pain Involves the Affective but not Sensory Components of Pain , 2004, Science.

[29]  Camelia M. Kuhnen,et al.  The Neural Basis of Financial Risk Taking , 2005, Neuron.

[30]  Colin Camerer,et al.  Neural Systems Responding to Degrees of Uncertainty in Human Decision-Making , 2005, Science.

[31]  Antonio Damasio,et al.  The somatic marker hypothesis: A neural theory of economic decision , 2005, Games Econ. Behav..

[32]  G. McCarthy,et al.  Decisions under Uncertainty: Probabilistic Context Influences Activation of Prefrontal and Parietal Cortices , 2005, The Journal of Neuroscience.

[33]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[34]  D. S. Zahm,et al.  Specificity in the Projections of Prefrontal and Insular Cortex to Ventral Striatopallidum and the Extended Amygdala , 2005, The Journal of Neuroscience.

[35]  J. Hirsch,et al.  A Neural Representation of Categorization Uncertainty in the Human Brain , 2006, Neuron.

[36]  S. Quartz,et al.  Neural Differentiation of Expected Reward and Risk in Human Subcortical Structures , 2006, Neuron.

[37]  Evan M. Gordon,et al.  Neural Signatures of Economic Preferences for Risk and Ambiguity , 2006, Neuron.

[38]  K. Preuschoff,et al.  Adding Prediction Risk to the Theory of Reward Learning , 2007, Annals of the New York Academy of Sciences.

[39]  Thomas Bengtsson,et al.  Sequential State and Variance Estimation within the Ensemble Kalman Filter , 2007 .

[40]  Justin S. Feinstein,et al.  Increased amygdala and insula activation during emotion processing in anxiety-prone subjects. , 2007, The American journal of psychiatry.

[41]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[42]  H. Damasio,et al.  Damage to the Insula Disrupts Addiction to Cigarette Smoking , 2007, Science.

[43]  E. Rolls,et al.  Cerebral Cortex Advance Access published June 22, 2007 Expected Value, Reward Outcome, and Temporal Difference Error Representations in a Probabilistic Decision Task , 2022 .