Computational Models of Performance Monitoring and Cognitive Control

The medial prefrontal cortex (mPFC) has been the subject of intense interest as a locus of cognitive control. Several computational models have been proposed to account for a range of effects, including error detection, conflict monitoring, error likelihood prediction, and numerous other effects observed with single-unit neurophysiology, fMRI, and lesion studies. Here, we review the state of computational models of cognitive control and offer a new theoretical synthesis of the mPFC as signaling response–outcome predictions. This new synthesis has two interacting components. The first component learns to predict the various possible outcomes of a planned action, and the second component detects discrepancies between the actual and intended responses; the detected discrepancies in turn update the outcome predictions. This single construct is consistent with a wide array of performance monitoring effects in mPFC and suggests a unifying account of the cognitive role of medial PFC in performance monitoring.

[1]  P. Haggard Human volition: towards a neuroscience of will , 2008, Nature Reviews Neuroscience.

[2]  Joshua W. Brown,et al.  Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding , 2003, Science.

[3]  Wayne D. Gray,et al.  An integrated model of cognitive control in task switching. , 2008, Psychological review.

[4]  Joshua W. Brown,et al.  Risk prediction and aversion by anterior cingulate cortex , 2007, Cognitive, affective & behavioral neuroscience.

[5]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[6]  Joshua W. Brown,et al.  Conflict effects without conflict in anterior cingulate cortex: Multiple response effects and context specific representations , 2009, NeuroImage.

[7]  H. Barbas,et al.  Synapses with Inhibitory Neurons Differentiate Anterior Cingulate from Dorsolateral Prefrontal Pathways Associated with Cognitive Control , 2009, Neuron.

[8]  Marco Steinhauser,et al.  Modeling behavioral measures of error detection in choice tasks: response monitoring versus conflict monitoring. , 2008, Journal of experimental psychology. Human perception and performance.

[9]  Jonathan D. Cohen,et al.  On the Control of Control: The Role of Dopamine in Regulating Prefrontal Function and Working Memory , 2007 .

[10]  Stephen Grossberg,et al.  How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades , 2004, Neural Networks.

[11]  A. Dale,et al.  Dorsal anterior cingulate cortex: A role in reward-based decision making , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[13]  Jonathan D. Cohen,et al.  The neural basis of error detection: conflict monitoring and the error-related negativity. , 2004, Psychological review.

[14]  Erik M. Altmann,et al.  Task switching and the pied homunculus: where are we being led? , 2003, Trends in Cognitive Sciences.

[15]  M. Botvinick,et al.  Error-likelihood prediction in the medial frontal cortex: a critical evaluation. , 2007, Cerebral cortex.

[16]  Joshua W. Brown,et al.  How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues , 1999, The Journal of Neuroscience.

[17]  G. Schoenbaum,et al.  Encoding Predicted Outcome and Acquired Value in Orbitofrontal Cortex during Cue Sampling Depends upon Input from Basolateral Amygdala , 2003, Neuron.

[18]  E. Rolls,et al.  Attention and working memory: a dynamical model of neuronal activity in the prefrontal cortex , 2003, The European journal of neuroscience.

[19]  Wayne D. Gray,et al.  Topics in Cognitive Science , 2009 .

[20]  William H. Alexander,et al.  Competition between learned reward and error outcome predictions in anterior cingulate cortex , 2010, NeuroImage.

[21]  V. B. Brooks,et al.  ‘Error’ potentials in limbic cortex (anterior cingulate area 24) of monkeys during motor learning , 1986, Neuroscience Letters.

[22]  M. Brass,et al.  To Do or Not to Do: The Neural Signature of Self-Control , 2007, The Journal of Neuroscience.

[23]  M. Coles,et al.  Performance monitoring in a confusing world: error-related brain activity, judgments of response accuracy, and types of errors. , 2000, Journal of experimental psychology. Human perception and performance.

[24]  J Schlag,et al.  Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. , 2000, Journal of neurophysiology.

[25]  T. Shallice,et al.  CONTENTION SCHEDULING AND THE CONTROL OF ROUTINE ACTIVITIES , 2000, Cognitive neuropsychology.

[26]  D. Durstewitz,et al.  A Neurocomputational Theory of the Dopaminergic Modulation of Working Memory Functions , 1999, The Journal of Neuroscience.

[27]  T. Braver,et al.  Anterior cingulate cortex and response conflict: effects of frequency, inhibition and errors. , 2001, Cerebral cortex.

[28]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[29]  N. Yeung,et al.  Anterior Cingulate Cortex , 2002 .

[30]  Todd S. Braver,et al.  A model of dual control mechanisms through anterior cingulate and prefrontal cortex interactions , 2006, Neurocomputing.

[31]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[32]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[33]  Joshua W. Brown,et al.  Computational and neural mechanisms of task switching , 2006, Neurocomputing.

[34]  Jeremy R. Reynolds,et al.  A computational model of fractionated conflict-control mechanisms in task-switching , 2007, Cognitive Psychology.

[35]  J. Hohnsbein,et al.  Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. , 1991, Electroencephalography and clinical neurophysiology.

[36]  J. Tanji,et al.  Role for cingulate motor area cells in voluntary movement selection based on reward. , 1998, Science.

[37]  Clay B. Holroyd,et al.  Hierarchical error processing: Different errors, different systems , 2007, Brain Research.

[38]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .

[39]  Norbert Kathmann,et al.  Neural correlates of error awareness , 2007, NeuroImage.

[40]  A. Jersild Mental set and shift , 2011 .

[41]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[42]  T. Sejnowski,et al.  Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. , 2000, Journal of neurophysiology.

[43]  Clay B. Holroyd,et al.  A mechanism for error detection in speeded response time tasks. , 2005, Journal of experimental psychology. General.

[44]  Lawrence R. Frank,et al.  Anterior cingulate activity modulates nonlinear decision weight function of uncertain prospects , 2006, NeuroImage.

[45]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[46]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[47]  David Goodman,et al.  Performance Monitoring in the Anterior Cingulate is Not All Error Related: Expectancy Deviation and the Representation of Action-Outcome Associations , 2007, Journal of Cognitive Neuroscience.

[48]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[49]  P. Goldman-Rakic,et al.  D1 Receptor in Interneurons of Macaque Prefrontal Cortex: Distribution and Subcellular Localization , 1998, The Journal of Neuroscience.

[50]  P. Rabbitt Errors and error correction in choice-response tasks. , 1966, Journal of experimental psychology.

[51]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[52]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[53]  D. Alan Allport,et al.  SHIFTING INTENTIONAL SET - EXPLORING THE DYNAMIC CONTROL OF TASKS , 1994 .

[54]  Ryan K. Jessup,et al.  Error Effects in Anterior Cingulate Cortex Reverse when Error Likelihood Is High , 2010, The Journal of Neuroscience.

[55]  Jonathan D. Wallis,et al.  Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables , 2009, Journal of Cognitive Neuroscience.

[56]  T. Kohonen Self-organized formation of topographically correct feature maps , 1982 .

[57]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[58]  Olaf Sporns,et al.  A Large-scale Neurocomputational Model of Task-oriented Behavior Selection and Working Memory in Prefrontal Cortex , 2006 .

[59]  James L. McClelland,et al.  On the control of automatic processes: a parallel distributed processing account of the Stroop effect. , 1990, Psychological review.

[60]  Javier Quintana,et al.  Anterior cingulate dysfunction during choice anticipation in schizophrenia , 2004, Psychiatry Research: Neuroimaging.

[61]  Yoshikazu Isomura,et al.  Neural Coding of “Attention for Action” and “Response Selection” in Primate Anterior Cingulate Cortex , 2003, The Journal of Neuroscience.

[62]  Michael J. Frank,et al.  Interactions between frontal cortex and basal ganglia in working memory: A computational model , 2001, Cognitive, affective & behavioral neuroscience.

[63]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[64]  Joshua W. Brown,et al.  A computational model of risk, conflict, and individual difference effects in the anterior cingulate cortex , 2008, Brain Research.

[65]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[66]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[67]  Clay B. Holroyd,et al.  The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback. , 2008, Psychophysiology.

[68]  Reza Shadmehr,et al.  Motor Learning and Memory for Reaching and Pointing , 2004 .

[69]  Tim Curran,et al.  Cross-task individual differences in error processing: Neural, electrophysiological, and genetic components , 2007, Cognitive, affective & behavioral neuroscience.

[70]  Carl R. Olson,et al.  Neuronal activity related to rule and conflict in macaque supplementary eye field , 2002, Physiology & Behavior.

[71]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[72]  Olaf Sporns,et al.  A Large-scale Neurocomputational Model of Task-oriented Behavior Selection and Working Memory in Prefrontal Cortex , 2006, Journal of Cognitive Neuroscience.

[73]  Stephen Monsell,et al.  Residual costs in task switching: Testing the failure-to-engage hypothesis , 2002, Psychonomic bulletin & review.

[74]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[75]  M. Botvinick Conflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function , 2007, Cognitive, affective & behavioral neuroscience.

[76]  John J. Foxe,et al.  The Anterior Cingulate and Error Avoidance , 2006, The Journal of Neuroscience.

[77]  Joshua W. Brown,et al.  Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex , 2005, Science.

[78]  A. Graybiel,et al.  Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[79]  C. Eriksen,et al.  Effects of noise letters upon the identification of a target letter in a nonsearch task , 1974 .

[80]  Matthew M. Botvinick,et al.  Anticipation of cognitive demand during decision-making , 2009, Psychological research.

[81]  David Badre,et al.  Computational and neurobiological mechanisms underlying cognitive flexibility. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[82]  Peter Dayan,et al.  Bee foraging in uncertain environments using predictive hebbian learning , 1995, Nature.

[83]  J. Houk,et al.  Model of cortical-basal ganglionic processing: encoding the serial order of sensory events. , 1998, Journal of neurophysiology.

[84]  Jonathan D. Cohen,et al.  Anterior Cingulate Cortex, Conflict Monitoring, and Levels of Processing , 2001, NeuroImage.

[85]  Mark V. Albert,et al.  Anticipation of conflict monitoring in the anterior cingulate cortex and the prefrontal cortex , 2007, Proceedings of the National Academy of Sciences.

[86]  Clay B. Holroyd,et al.  ERP correlates of feedback and reward processing in the presence and absence of response choice. , 2005, Cerebral cortex.

[87]  A. Pavlovic,et al.  The anterior cingulate cortex , 2009 .

[88]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[89]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[90]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[91]  W. Schultz,et al.  Neuronal activity in monkey ventral striatum related to the expectation of reward , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[92]  T. Braver,et al.  Explaining the many varieties of working memory variation: Dual mechanisms of cognitive control. , 2007 .

[93]  Clay B. Holroyd,et al.  Electroencephalographic correlates of target and outcome errors , 2008, Experimental Brain Research.

[94]  Masataka Watanabe,et al.  Prefrontal and cingulate unit activity during timing behavior in the monkey , 1979, Brain Research.

[95]  William H. Alexander,et al.  Shifting Attention Using a Temporal Difference Prediction Error and High-Dimensional Input , 2007, Adapt. Behav..

[96]  J. Ridley Studies of Interference in Serial Verbal Reactions , 2001 .

[97]  D. Norman,et al.  Attention to Action: Willed and Automatic Control of Behavior Technical Report No. 8006. , 1980 .

[98]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[99]  S. Monsell,et al.  Costs of a predictible switch between simple cognitive tasks. , 1995 .

[100]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[101]  T. Shallice,et al.  Task Switching : A PDP Model , 2001 .

[102]  Tomas C. Bellamy,et al.  On the activation of soluble guanylyl cyclase by nitric oxide , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[103]  J. Cohen,et al.  Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. , 2000, Science.

[104]  B. Silvano Zanutto,et al.  Dopamine and Norepinephrine Modulation of Cortical and Subcortical Dynamics During Visuomotor Learning , 2007 .

[105]  D. Laming Choice reaction performance following an error , 1979 .

[106]  Richard B. Ivry,et al.  Task switching and multitask performance. , 2000 .

[107]  Keiji Tanaka,et al.  Neuronal Correlates of Goal-Based Motor Selection in the Prefrontal Cortex , 2003, Science.

[108]  P. Goldman-Rakic,et al.  D 1 Receptor in Interneurons of Macaque Prefrontal Cortex : Distribution and Subcellular Localization , 1998 .

[109]  E. Aarts,et al.  Anticipatory activity in anterior cingulate cortex can be independent of conflict and error likelihood. , 2008, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[110]  Bruce D. McCandliss,et al.  Response Anticipation and Response Conflict: An Event-Related Potential and Functional Magnetic Resonance Imaging Study , 2007, The Journal of Neuroscience.

[111]  Jonathan D. Cohen,et al.  A computational model of anterior cingulate function in speeded response tasks: Effects of frequency, sequence, and conflict , 2002, Cognitive, affective & behavioral neuroscience.

[112]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[113]  Jan Seifert,et al.  The relation of cortical activity and BIS/BAS on the trait level , 2006, Biological Psychology.

[114]  Clay B. Holroyd,et al.  Reward prediction error signals associated with a modified time estimation task. , 2007, Psychophysiology.

[115]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[116]  E. Procyk,et al.  Anterior cingulate activity during routine and non-routine sequential behaviors in macaques , 2000, Nature Neuroscience.

[117]  Jonathan D. Cohen,et al.  Conflict monitoring versus selection-for-action in anterior cingulate cortex , 1999, Nature.

[118]  B. Vogt,et al.  Contributions of anterior cingulate cortex to behaviour. , 1995, Brain : a journal of neurology.

[119]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[120]  E. Rolls,et al.  Neurodynamics of biased competition and cooperation for attention: a model with spiking neurons. , 2005, Journal of neurophysiology.

[121]  M. Delgado,et al.  The role of the striatum in aversive learning and aversive prediction errors , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[122]  Marie T. Banich,et al.  Neural Mechanisms of Cognitive Control: An Integrative Model of Stroop Task Performance and fMRI Data , 2006, Journal of Cognitive Neuroscience.