Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions

The capacity to accurately evaluate the causal effectiveness of our actions is key to successfully adapting to changing environments. Here we scanned subjects using functional magnetic resonance imaging while they pressed a button to earn money as the response–reward relationship changed over time. Subjects' judgments about the causal efficacy of their actions reflected the objective contingency between the rate of button pressing and the amount of money they earned. Neural responses in medial orbitofrontal cortex and dorsomedial striatum were modulated as a function of contingency, by increasing in activity during sessions when actions were highly causal compared with when they were not. Moreover, medial prefrontal cortex tracked local changes in action–outcome correlations, implicating this region in the on-line computation of contingency. These results reveal the involvement of distinct brain regions in the computational processes that establish the causal efficacy of actions, providing insight into the neural mechanisms underlying the adaptive control of behavior.

[1]  W M Baum,et al.  The correlation-based law of effect. , 1973, Journal of the experimental analysis of behavior.

[2]  L. J. Hammond The effect of contingency upon the appetitive conditioning of free-operant behavior. , 1980, Journal of the experimental analysis of behavior.

[3]  B. Milner Some cognitive effects of frontal-lobe lesions in man. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[4]  Christopher D. Adams,et al.  The Effect of the Instrumental Training Contingency on Susceptibility to Reinforcer Devaluation , 1983 .

[5]  A. Dickinson,et al.  Performance on Ratio and Interval Schedules with Matched Reinforcement Rates , 1990, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[6]  B. Balleine,et al.  Actions and responses: The dual psychology of behaviour. , 1993 .

[7]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[8]  E. Rolls,et al.  Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. , 1994, Journal of neurology, neurosurgery, and psychiatry.

[9]  A. Damasio,et al.  Insensitivity to future consequences following damage to human prefrontal cortex , 1994, Cognition.

[10]  C. Frith,et al.  Prefrontal dysfunction in depressed patients performing a complex planning task: a study using positron emission tomography , 1997, Psychological Medicine.

[11]  G. Schoenbaum,et al.  Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning , 1998, Nature Neuroscience.

[12]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[13]  B. Balleine,et al.  The Effect of Lesions of the Insular Cortex on Instrumental Conditioning: Evidence for a Role in Incentive Memory , 2000, The Journal of Neuroscience.

[14]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[15]  M. Raichle,et al.  Searching for a baseline: Functional imaging and the resting human brain , 2001, Nature Reviews Neuroscience.

[16]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[17]  J. O'Doherty,et al.  Appetitive and Aversive Olfactory Learning in Humans Studied Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[18]  Brian Knutson,et al.  A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI , 2003, NeuroImage.

[19]  J. O'Doherty,et al.  Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex , 2003, Science.

[20]  B. Balleine,et al.  The role of prelimbic cortex in instrumental conditioning , 2003, Behavioural Brain Research.

[21]  Keiji Tanaka,et al.  Neuronal Correlates of Goal-Based Motor Selection in the Prefrontal Cortex , 2003, Science.

[22]  J. Hollerman,et al.  Changes in behavior-related neuronal activity in the striatum during learning , 2003, Trends in Neurosciences.

[23]  S. Killcross,et al.  Coordination of actions and habits in the medial prefrontal cortex of rats. , 2003, Cerebral cortex.

[24]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[25]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[26]  B. Balleine Neural bases of food-seeking: Affect, arousal and reward in corticostriatolimbic circuits , 2005, Physiology & Behavior.

[27]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[28]  S. Inati,et al.  An fMRI study of reward-related probability learning , 2005, NeuroImage.

[29]  Joseph J. Paton,et al.  The primate amygdala represents the positive and negative value of visual stimuli during learning , 2006, Nature.

[30]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[31]  Human contingency learning : recent trends in research and theory , 2007 .

[32]  Vivian V. Valentin,et al.  Determining the Neural Substrates of Goal-Directed Learning in the Human Brain , 2007, The Journal of Neuroscience.

[33]  B. Balleine,et al.  Still at the Choice‐Point , 2007, Annals of the New York Academy of Sciences.

[34]  B. Balleine,et al.  Multiple Forms of Value Learning and the Function of Dopamine , 2009 .

[35]  A. Dickinson Instrumental Conditioning , 2020, Encyclopedia of Evolutionary Psychological Science.