The Impact of Deliberative Strategy Dissociates ERP Components Related to Conflict Processing vs. Reinforcement Learning

We applied the event-related brain potential (ERP) technique to investigate the involvement of two neuromodulatory systems in learning and decision making: The locus coeruleus–norepinephrine system (NE system) and the mesencephalic dopamine system (DA system). We have previously presented evidence that the N2, a negative deflection in the ERP elicited by task-relevant events that begins approximately 200 ms after onset of the eliciting stimulus and that is sensitive to low-probability events, is a manifestation of cortex-wide noradrenergic modulation recruited to facilitate the processing of unexpected stimuli. Further, we hold that the impact of DA reinforcement learning signals on the anterior cingulate cortex (ACC) produces a component of the ERP called the feedback-related negativity (FRN). The N2 and the FRN share a similar time range, a similar topography, and similar antecedent conditions. We varied factors related to the degree of cognitive deliberation across a series of experiments to dissociate these two ERP components. Across four experiments we varied the demand for a deliberative strategy, from passively watching feedback, to more complex/challenging decision tasks. Consistent with our predictions, the FRN was largest in the experiment involving active learning and smallest in the experiment involving passive learning whereas the N2 exhibited the opposite effect. Within each experiment, when subjects attended to color, the N2 was maximal at frontal–central sites, and when they attended to gender it was maximal over lateral-occipital areas, whereas the topology of the FRN was frontal–central in both task conditions. We conclude that both the DA system and the NE system act in concert when learning from rewards that vary in expectedness, but that the DA system is relatively more exercised when subjects are relatively more engaged by the learning task.

[1]  Jonathan D. Cohen,et al.  Phasic Activation of Monkey Locus Ceruleus Neurons by Simple Decisions in a Forced-Choice Task , 2004, The Journal of Neuroscience.

[2]  G. McCarthy,et al.  On the influence of task relevance and stimulus probability on event-related-potential components. , 1977, Electroencephalography and clinical neurophysiology.

[3]  Clay B. Holroyd,et al.  It's worse than you thought: the feedback negativity and violations of reward prediction in gambling tasks. , 2007, Psychophysiology.

[4]  Clay B. Holroyd,et al.  The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback. , 2008, Psychophysiology.

[5]  Terrence J. Sejnowski,et al.  Choice modulates the neural dynamics of prediction error processing during rewarded learning , 2011, NeuroImage.

[6]  P. Michie,et al.  The attentional blink and P300. , 1999, Neuroreport.

[7]  K L Shapiro,et al.  Temporary suppression of visual processing in an RSVP task: an attentional blink? . , 1992, Journal of experimental psychology. Human perception and performance.

[8]  Clay B. Holroyd,et al.  ERP correlates of feedback and reward processing in the presence and absence of response choice. , 2005, Cerebral cortex.

[9]  E Donchin,et al.  A new method for off-line removal of ocular artifact. , 1983, Electroencephalography and clinical neurophysiology.

[10]  N. Squires,et al.  The effect of stimulus sequence on the waveform of the cortical event-related potential. , 1976, Science.

[11]  Clay B. Holroyd,et al.  A note on the oddball N200 and the feedback ERN , 2004 .

[12]  M. Masson,et al.  Target—distractor interference in the attentional blink implicates the locus coeruleus—norepinephrine system , 2009, Psychonomic bulletin & review.

[13]  Clay B. Holroyd,et al.  A mechanism for error detection in speeded response time tasks. , 2005, Journal of experimental psychology. General.

[14]  W. Ritter,et al.  The scalp topography of potentials associated with missing visual or auditory stimuli. , 1976, Electroencephalography and clinical neurophysiology.

[15]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[16]  C. Holroyd,et al.  Which way do I go? Neural activation in response to feedback and spatial processing in a virtual T-maze. , 2009, Cerebral cortex.

[17]  Jonathan R. Folstein,et al.  Influence of cognitive control and mismatch on the N2 component of the ERP: a review. , 2007, Psychophysiology.

[18]  G. Aston-Jones,et al.  Activation of monkey locus coeruleus neurons varies with difficulty and performance in a target detection task. , 2004, Journal of neurophysiology.

[19]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[20]  E. Courchesne,et al.  Stimulus novelty, task relevance and the visual evoked potential in man. , 1975, Electroencephalography and clinical neurophysiology.

[21]  Clay B. Holroyd,et al.  Errors in reward prediction are re£ected in the event-related brain potential , 2003 .

[22]  E. Donchin,et al.  On quantifying surprise: the variation of event-related potentials with subjective probability. , 1977, Psychophysiology.

[23]  Jonathan D. Cohen,et al.  Simplified dynamics in a model of noradrenergic modulation of cognitive performance , 2002, Neural Networks.

[24]  Clay B. Holroyd,et al.  What can topology changes in the oddball N2 reveal about underlying processes? , 2011, Neuroreport.

[25]  J. Pineda,et al.  Effects of locus coeruleus lesions on auditory, long-latency, event- related potentials in monkey , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[27]  R. Johnson,et al.  On the neural generators of the P300 component of the event-related potential. , 2007, Psychophysiology.

[28]  S. L. Foote,et al.  Effects of systemic clonidine on auditory event-related potentials in squirrel monkeys , 1994, Brain Research Bulletin.

[29]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[30]  N. Squires,et al.  Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. , 1975, Electroencephalography and clinical neurophysiology.

[31]  P. H. Lindsay,et al.  Evoked Potential Correlates of Auditory Signal Detection , 1971, Science.

[32]  Mark S. Gilzenrat,et al.  The role of the locus coeruleus in mediating the attentional blink: a neurocomputational theory. , 2005, Journal of experimental psychology. General.

[33]  K. R. Ridderinkhof,et al.  Electrophysiological correlates of anterior cingulate function in a go/no-go task: Effects of response conflict and trial type frequency , 2003, Cognitive, affective & behavioral neuroscience.

[34]  Phillip J. Holcomb,et al.  The Influence of Stimulus Deviance on Electrophysiologic and Behavioral Responses to Novel Events , 2000, Journal of Cognitive Neuroscience.

[35]  Adrian R. Willoughby,et al.  The Medial Frontal Cortex and the Rapid Processing of Monetary Gains and Losses , 2002, Science.

[36]  C. Berridge,et al.  The locus coeruleus–noradrenergic system: modulation of behavioral state and state-dependent cognitive processes , 2003, Brain Research Reviews.

[37]  D Friedman,et al.  A brain event related to the making of a sensory discrimination. , 1979, Science.

[38]  Clay B. Holroyd,et al.  Reward prediction error signals associated with a modified time estimation task. , 2007, Psychophysiology.

[39]  S. Nieuwenhuis,et al.  Investigating the role of the noradrenergic system in human cognition , 2011 .

[40]  M. Westerfield,et al.  Monkey P3 in an “oddball” paradigm: Pharmacological support for multiple neural sources , 1993, Brain Research Bulletin.

[41]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[42]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[43]  B S Kopell,et al.  Auditory evoked potentials to unpredictable shifts in pitch. , 1976, Psychophysiology.

[44]  Emanuel Donchin,et al.  The P300 component of the event-related brain potential as an index of information processing , 1982, Biological Psychology.

[45]  E Callaway,et al.  The effect of D-amphetamine, clonidine, and yohimbine on human information processing. , 1994, Psychophysiology.

[46]  Clay B. Holroyd,et al.  Electroencephalographic correlates of target and outcome errors , 2008, Experimental Brain Research.

[47]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[48]  Johannes Hewig,et al.  Decision-making in Blackjack: an electrophysiological analysis. , 2007, Cerebral cortex.

[49]  Jonathan R. Folstein,et al.  Novelty and conflict in the categorization of complex stimuli. , 2008, Psychophysiology.

[50]  J. Cohen,et al.  The role of locus coeruleus in the regulation of cognitive performance. , 1999, Science.

[51]  Dick J. Veltman,et al.  Effects of the noradrenergic agonist clonidine on temporal and spatial attention , 2007, Psychopharmacology.

[52]  M M Mesulam,et al.  An electrophysiological index of stimulus unfamiliarity. , 2000, Psychophysiology.

[53]  S. Nieuwenhuis,et al.  The anatomical and functional relationship between the P3 and autonomic components of the orienting response. , 2011, Psychophysiology.

[54]  S. L. Foote,et al.  Impulse conduction properties of noradrenergic locus coeruleus axons projecting to monkey cerebrocortex , 1985, Neuroscience.

[55]  Jonathan D. Cohen,et al.  The neural basis of error detection: conflict monitoring and the error-related negativity. , 2004, Psychological review.

[56]  Clay B. Holroyd,et al.  Responsibility modulates neural mechanisms of outcome processing: an ERP study. , 2011, Psychophysiology.

[57]  R. Baker,et al.  When is an error not a prediction error? An electrophysiological investigation , 2009, Cognitive, affective & behavioral neuroscience.

[58]  C. Carter,et al.  The Timing of Action-Monitoring Processes in the Anterior Cingulate Cortex , 2002, Journal of Cognitive Neuroscience.

[59]  Clay B. Holroyd,et al.  Implementation of error-processing in the human anterior cingulate cortex: a source analysis of the magnetic equivalent of the error-related negativity , 2003, Biological Psychology.

[60]  C. Carter,et al.  The anterior cingulate as a conflict monitor: fMRI and ERP studies , 2002, Physiology & Behavior.

[61]  Clay B. Holroyd,et al.  Dissociated roles of the anterior cingulate cortex in reward and conflict processing as revealed by the feedback error-related negativity and N200 , 2011, Biological Psychology.

[62]  Jonathan D. Cohen,et al.  Decision making, the P3, and the locus coeruleus-norepinephrine system. , 2005, Psychological bulletin.

[63]  J D Cohen,et al.  A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. , 1990, Science.