Mediofrontal Negativity Signals Unexpected Timing of Salient Outcomes

The medial prefrontal cortex (mPFC) and ACC have been consistently implicated in learning predictions of future outcomes and signaling prediction errors (i.e., unexpected deviations from such predictions). A computational model of ACC/mPFC posits that these prediction errors should be modulated by outcomes occurring at unexpected times, even if the outcomes themselves are predicted. However, unexpectedness per se is not the only variable that modulates ACC/mPFC activity, as studies reported its sensitivity to the salience of outcomes. In this study, mediofrontal negativity, a component of the event-related brain potential generated in ACC/mPFC and coding for prediction errors, was measured in 48 participants performing a Pavlovian aversive conditioning task, during which aversive (thus salient) and neutral outcomes were unexpectedly shifted (i.e., anticipated or delayed) in time. Mediofrontal ERP signals of prediction error were observed for outcomes occurring at unexpected times but were specific for salient (shock-associated), as compared with neutral, outcomes. These findings have important implications for the theoretical accounts of ACC/mPFC and suggest a critical role of timing and salience information in prediction error signaling.

[1]  Deborah Talmi,et al.  The Feedback-Related Negativity Signals Salience Prediction Errors, Not Reward Prediction Errors , 2013, The Journal of Neuroscience.

[2]  M. Roesch,et al.  Neuronal Activity Related to Reward Value and Motivation in Primate Frontal Cortex , 2004, Science.

[3]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[4]  Masataka Watanabe,et al.  Prefrontal and cingulate unit activity during timing behavior in the monkey , 1979, Brain Research.

[5]  Jutta Peterburs,et al.  Feedback delay gradually affects amplitude and valence specificity of the feedback-related negativity (FRN). , 2016, Psychophysiology.

[6]  P. Redgrave,et al.  Is the short-latency dopamine response too short to signal reward error? , 1999, Trends in Neurosciences.

[7]  Joseph E LeDoux,et al.  Evidence for recovery of fear following immediate extinction in rats and humans. , 2008, Learning & memory.

[8]  P. Dayan,et al.  Reinforcement learning: The Good, The Bad and The Ugly , 2008, Current Opinion in Neurobiology.

[9]  G. Pagnoni,et al.  Human Striatal Responses to Monetary Reward Depend On Saliency , 2004, Neuron.

[10]  David Goodman,et al.  Performance Monitoring in the Anterior Cingulate is Not All Error Related: Expectancy Deviation and the Representation of Action-Outcome Associations , 2007, Journal of Cognitive Neuroscience.

[11]  A. Nobre,et al.  Where and When to Pay Attention: The Neural Systems for Directing Attention to Spatial Locations and to Time Intervals as Revealed by Both PET and fMRI , 1998, The Journal of Neuroscience.

[12]  William H. Alexander,et al.  Hierarchical Error Representation: A Computational Model of Anterior Cingulate and Dorsolateral Prefrontal Cortex , 2015, Neural Computation.

[13]  Nicola K. Ferdinand,et al.  The Processing of Unexpected Positive Response Outcomes in the Mediofrontal Cortex , 2012, The Journal of Neuroscience.

[14]  Better late than never? The effect of feedback delay on ERP indices of reward processing , 2012, Cognitive, affective & behavioral neuroscience.

[15]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[16]  P. Goldman-Rakic,et al.  Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. , 1993, Cerebral cortex.

[17]  A. Sanfey,et al.  Independent Coding of Reward Magnitude and Valence in the Human Brain , 2004, The Journal of Neuroscience.

[18]  M. Walton,et al.  Separate neural pathways process different decision costs , 2006, Nature Neuroscience.

[19]  N. Yeung,et al.  On the ERN and the significance of errors. , 2005, Psychophysiology.

[20]  Joshua W. Brown,et al.  Medial prefrontal cortex as an action-outcome predictor , 2011, Nature Neuroscience.

[21]  A. Engel,et al.  Trial-by-Trial Coupling of Concurrent Electroencephalogram and Functional Magnetic Resonance Imaging Identifies the Dynamics of Performance Monitoring , 2005, The Journal of Neuroscience.

[22]  Marco Steinhauser,et al.  Error-related brain activity and adjustments of selective attention following errors , 2011, NeuroImage.

[23]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[24]  I. THE ATTENTION SYSTEM OF THE HUMAN BRAIN , 2002 .

[25]  David S. Touretzky,et al.  Representation and Timing in Theories of the Dopamine System , 2006, Neural Computation.

[26]  P. Sajda,et al.  Perceptual Salience and Reward Both Influence Feedback-Related Neural Activity Arising from Choice , 2015, The Journal of Neuroscience.

[27]  R. Rescorla A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .

[28]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[29]  Timothy E. J. Behrens,et al.  Dissociable Reward and Timing Signals in Human Midbrain and Ventral Striatum , 2011, Neuron.

[30]  R. Davidson,et al.  The integration of negative affect, pain and cognitive control in the cingulate cortex , 2011, Nature Reviews Neuroscience.

[31]  Clay B. Holroyd,et al.  The feedback-related negativity reflects the binary evaluation of good versus bad outcomes , 2006, Biological Psychology.

[32]  John R. Anderson,et al.  Learning from experience: Event-related potential correlates of reward processing, neural adaptation, and behavioral choice , 2012, Neuroscience & Biobehavioral Reviews.

[33]  Terrence J. Sejnowski,et al.  Blind separation and blind deconvolution: an information-theoretic approach , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[34]  Lesya Y. Ganushchak,et al.  Motivation and semantic context affect brain error-monitoring activity: An event-related brain potentials study , 2008, NeuroImage.

[35]  G. Glover,et al.  Dissociable Intrinsic Connectivity Networks for Salience Processing and Executive Control , 2007, The Journal of Neuroscience.

[36]  Martin E. Maier,et al.  Updating Expected Action Outcome in the Medial Frontal Cortex Involves an Evaluation of Error Type , 2013, The Journal of Neuroscience.

[37]  N. Mackintosh A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[38]  Jeremy Goslin,et al.  Principal components analysis of reward prediction errors in a reinforcement learning task , 2016, NeuroImage.

[39]  Ryan K. Jessup,et al.  Error Effects in Anterior Cingulate Cortex Reverse when Error Likelihood Is High , 2010, The Journal of Neuroscience.

[40]  Jeff T. Larsen,et al.  The good, the bad and the neutral: Electrophysiological responses to feedback stimuli , 2006, Brain Research.

[41]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[42]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[43]  Clay B. Holroyd,et al.  Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance , 2004, Neuroscience & Biobehavioral Reviews.

[44]  Keiji Tanaka,et al.  Medial prefrontal cell activity signaling prediction errors of action values , 2007, Nature Neuroscience.

[45]  Tobias U. Hauser,et al.  The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization , 2014, NeuroImage.

[46]  Martin E. Maier,et al.  Mediofrontal negativity signals unexpected omission of aversive events , 2014, Scientific Reports.

[47]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[48]  Marco Steinhauser,et al.  Enhanced error-related negativity on flanker errors: error expectancy or error significance? , 2012, Psychophysiology.

[49]  Massimo Silvetti,et al.  Value and prediction error estimation account for volatility effects in ACC: A model-based fMRI study , 2013, Cortex.

[50]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[51]  Michael X. Cohen,et al.  It's about Time , 2010, Front. Hum. Neurosci.

[52]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[53]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[54]  R. Rescorla,et al.  Spatial contiguity facilitates Pavlovian second-order conditioning. , 1979, Journal of experimental psychology. Animal behavior processes.

[55]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[56]  J Schlag,et al.  Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. , 2000, Journal of neurophysiology.

[57]  P300, not feedback error-related negativity, manifests the waiting cost of receiving reward information , 2014, Neuroreport.

[58]  L. Becerra,et al.  Neural circuitry underlying pain modulation: expectation, hypnosis, placebo , 2003, Trends in Cognitive Sciences.

[59]  Tom Eichele,et al.  Semi-automatic identification of independent components representing EEG artifact , 2009, Clinical Neurophysiology.

[60]  J. Horvitz Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events , 2000, Neuroscience.

[61]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[62]  Terrence J. Sejnowski,et al.  Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis , 2007, NeuroImage.

[63]  William H. Alexander,et al.  A general role for medial prefrontal cortex in event prediction , 2014, Front. Comput. Neurosci..

[64]  Timothy E. Ham,et al.  Cognitive Control and the Salience Network: An Investigation of Error Processing and Effective Connectivity , 2013, The Journal of Neuroscience.

[65]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[66]  John-Dylan Haynes,et al.  Disentangling neural representations of value and salience in the human brain , 2014, Proceedings of the National Academy of Sciences.

[67]  Joshua W. Brown,et al.  Medial prefrontal cortex predicts and evaluates the timing of action outcomes , 2011, NeuroImage.