The costs and benefits of brain dopamine for cognitive control.

Cognitive control helps us attain our goals by resisting distraction and temptations. Dopaminergic drugs are well known to enhance cognitive control. However, there is great variability in the effects of dopaminergic drugs across different contexts, with beneficial effects on some tasks but detrimental effects on other tasks. The mechanisms underlying this variability across cognitive task demands remain unclear. I aim to elucidate this across-task variability in dopaminergic drug efficacy by going beyond classic models that emphasize the importance of dopamine in the prefrontal cortex for cognitive control and working memory. To this end, I build on recent advances in cognitive neuroscience that highlight a role for dopamine in cost-benefit decision making. Specifically, I reconceptualize cognitive control as involving not just prefrontal dopamine but also modulation of cost-benefit decision making by striatal dopamine. This approach will help us understand why we sometimes fail to (choose to) exert cognitive control while also identifying mechanistic factors that predict dopaminergic drug effects on cognitive control. WIREs Cogn Sci 2016, 7:317-329. doi: 10.1002/wcs.1401 For further resources related to this article, please visit the WIREs website.

[1]  W. Schultz.  Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[2]  P. Goldman-Rakic,et al.  Dopamine D1 receptor mechanisms in the cognitive performance of young adult and aged monkeys , 1994, Psychopharmacology.

[3]  R. Cools,et al.  Establishing the dopamine dependency of human striatal signals during reward and punishment reversal learning. , 2014, Cerebral cortex.

[4]  P. Dayan,et al.  Dopaminergic Modulation of Decision Making and Subjective Well-Being , 2015, The Journal of Neuroscience.

[5]  M. Liechti,et al.  Swiss University Students’ Attitudes toward Pharmacological Cognitive Enhancement , 2015, PloS one.

[6]  M. Inzlicht,et al.  Why self-control seems (but may not be) limited , 2014, Trends in Cognitive Sciences.

[7]  R. Dolan,et al.  Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making , 2015, Proceedings of the National Academy of Sciences.

[8]  N. Daw,et al.  Model-based learning protects against forming habits , 2015, Cognitive, Affective, & Behavioral Neuroscience.

[9]  S. Floresco.  Prefrontal dopamine and behavioral flexibility: shifting from an “inverted-U” toward a family of functions , 2013, Front. Neurosci..

[10]  P. Goldman-Rakic,et al.  D1 dopamine receptors in prefrontal cortex: involvement in working memory , 1991, Science.

[11]  T. Robbins,et al.  Differential effects of 6-OHDA lesions of the frontal cortex and caudate nucleus on the ability to acquire an attentional set. , 2001, Cerebral cortex.

[12]  T. Robinson,et al.  A selective role for dopamine in reward learning , 2010, Nature.

[13]  P. Dayan.  How to set the switches on this thing , 2012, Current Opinion in Neurobiology.

[14]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[15]  M. Botvinick,et al.  A labor/leisure tradeoff in cognitive control. , 2014, Journal of experimental psychology. General.

[16]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[17]  Angela L. Duckworth,et al.  An opportunity cost model of subjective effort and task performance. , 2013, The Behavioral and brain sciences.

[18]  Ulrik R Beierholm,et al.  Dopamine Modulates Reward-Related Vigor , 2013, Neuropsychopharmacology.

[19]  B. Sahakian,et al.  The increasing lifestyle use of modafinil by healthy people: safety and ethical issues , 2015, Current Opinion in Behavioral Sciences.

[20]  J. Seamans,et al.  The principal features and mechanisms of dopamine modulation in the prefrontal cortex , 2004, Progress in Neurobiology.

[21]  Kimberly S. Chiew,et al.  Mechanisms of motivation–cognition interaction: challenges and opportunities , 2014, Cognitive, affective & behavioral neuroscience.

[22]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[23]  G. Robert J. Hockey,et al.  A motivational control theory of cognitive fatigue. , 2011 .

[24]  N. Daw,et al.  Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions , 2011, Neuropsychopharmacology.

[25]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  Roshan Cools,et al.  Habitual versus Goal-directed Action Control in Parkinson Disease , 2011, Journal of Cognitive Neuroscience.

[27]  M. Frank,et al.  From reinforcement learning models to psychiatric and neurological disorders , 2011, Nature Neuroscience.

[28]  S. Floresco,et al.  Dopamine Antagonism Decreases Willingness to Expend Physical, But Not Cognitive, Effort: A Comparison of Two Rodent Cost/Benefit Decision-Making Tasks , 2015, Neuropsychopharmacology.

[29]  M. Khamassi,et al.  Dopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia , 2012, Front. Neurosci..

[30]  N. Daw,et al.  Dopamine selectively remediates 'model-based' reward learning: a computational approach. , 2016, Brain : a journal of neurology.

[31]  K. Berridge,et al.  Intra-Accumbens Amphetamine Increases the Conditioned Incentive Salience of Sucrose Reward: Enhancement of Reward “Wanting” without Enhanced “Liking” or Response Reinforcement , 2000, The Journal of Neuroscience.

[32]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[33]  M. Hasselmo,et al.  Modes and Models of Forebrain Cholinergic Neuromodulation of Cognition , 2011, Neuropsychopharmacology.

[34]  M D'Esposito,et al.  Enhanced frontal function in Parkinson's disease. , 2010, Brain : a journal of neurology.

[35]  M. Walton,et al.  Calculating utility: preclinical evidence for cost–benefit analysis by mesolimbic dopamine , 2007, Psychopharmacology.

[36]  Deanna L. Wallace,et al.  Dopamine and the Cognitive Downside of a Promised Bonus , 2014, Psychological science.

[37]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[38]  T. Braver,et al.  Cognitive effort: A neuroeconomic approach , 2015, Cognitive, affective & behavioral neuroscience.

[39]  E. Koechlin,et al.  Executive control and decision-making in the prefrontal cortex , 2015, Current Opinion in Behavioral Sciences.

[40]  G. Fernández,et al.  Dynamic adaptation of large-scale brain networks in response to acute stressors , 2014, Trends in Neurosciences.

[41]  J D Cohen,et al.  Multitasking versus multiplexing: Toward a normative account of limitations in the simultaneous execution of control-demanding behaviors , 2014, Cognitive, affective & behavioral neuroscience.

[42]  Jonathan D. Power,et al.  Multi-task connectivity reveals flexible hubs for adaptive task control , 2013, Nature Neuroscience.

[43]  Wolfgang Hauber,et al.  Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer. , 2008, Learning & memory.

[44]  Alice Y. Chiang,et al.  Working-memory capacity protects model-based learning from stress , 2013, Proceedings of the National Academy of Sciences.

[45]  P. Dayan,et al.  Action controls dopaminergic enhancement of reward representations , 2012, Proceedings of the National Academy of Sciences.

[46]  J. Salamone,et al.  Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits , 2007, Psychopharmacology.

[47]  N. Daw,et al.  Deciding How To Decide: Self-Control and Meta-Decision Making , 2015, Trends in Cognitive Sciences.

[48]  Anne G E Collins,et al.  Opponent actor learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. , 2014, Psychological review.

[49]  M. D’Esposito,et al.  Reversal learning in Parkinson's disease depends on medication status and outcome valence , 2006, Neuropsychologia.

[50]  J. Mirenowicz,et al.  Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. , 2000, Behavioral neuroscience.

[51]  C. Winstanley,et al.  Sensitivity to Cognitive Effort Mediates Psychostimulant Effects on a Novel Rodent Cost/Benefit Decision-Making Task , 2012, Neuropsychopharmacology.

[52]  Michael W. Cole,et al.  Reward Motivation Enhances Task Coding in Frontoparietal Cortex. , 2016, Cerebral cortex.

[53]  C. Berridge,et al.  The Cognition-Enhancing Effects of Psychostimulants Involve Direct Action in the Prefrontal Cortex , 2015, Biological Psychiatry.

[54]  B. Sahakian,et al.  Professor's little helper , 2007, Nature.

[55]  T. Robbins,et al.  Dissociation in Effects of Lesions of the Nucleus Accumbens Core and Shell on Appetitive Pavlovian Approach Behavior and the Potentiation of Conditioned Reinforcement and Locomotor Activity byd-Amphetamine , 1999, The Journal of Neuroscience.

[56]  Roshan Cools,et al.  Anatomical connection strength predicts dopaminergic drug effects on fronto-striatal function , 2013, Psychopharmacology.

[57]  S. Floresco,et al.  Multiple Dopamine Receptor Subtypes in the Medial Prefrontal Cortex of the Rat Regulate Set-Shifting , 2006, Neuropsychopharmacology.

[58]  M. Naoi,et al.  The metabolism of l-DOPA and l-threo-3,4-dihydroxyphenylserine and their effects on monoamines in the human brain: analysis of the intraventricular fluid from parkinsonian patients , 1996, Journal of the Neurological Sciences.

[59]  P. Dayan,et al.  Action versus valence in decision making , 2014, Trends in Cognitive Sciences.

[60]  R. Klein,et al.  Psychostimulants Act Within the Prefrontal Cortex to Improve Cognitive Function , 2012, Biological Psychiatry.

[61]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[62]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[63]  N. Sigala,et al.  Dynamic Coding for Cognitive Control in Prefrontal Cortex , 2013, Neuron.

[64]  M. D’Esposito,et al.  Impulsive Personality Predicts Dopamine-Dependent Changes in Frontostriatal Activity during Component Processes of Working Memory , 2007, The Journal of Neuroscience.

[65]  T. Robbins,et al.  A role for mesencephalic dopamine in activation: commentary on Berridge (2006) , 2007, Psychopharmacology.

[66]  JaneR . Taylor,et al.  Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. , 2007, Cerebral cortex.

[67]  B. Milner.  Effects of Different Brain Lesions on Card Sorting: The Role of the Frontal Lobes , 1963 .

[68]  R. Cools,et al.  Selective attentional enhancement and inhibition of fronto-posterior connectivity by the basal ganglia during attention switching. , 2015, Cerebral cortex.

[69]  D. Durstewitz,et al.  The Dual-State Theory of Prefrontal Cortex Dopamine Function with Relevance to Catechol-O-Methyltransferase Genotypes and Schizophrenia , 2008, Biological Psychiatry.

[70]  T. Sejnowski,et al.  Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. , 2000, Journal of neurophysiology.

[71]  T. Braver,et al.  Dopamine Does Double Duty in Motivating Cognitive Effort , 2016, Neuron.

[72]  P. Dayan,et al.  Differential, but not opponent, effects of l-DOPA and citalopram on action learning with reward and punishment , 2013, Psychopharmacology.

[73]  Kartik K. Sreenivasan,et al.  Revisiting the role of persistent neural activity during working memory , 2014, Trends in Cognitive Sciences.

[74]  Timothy E. J. Behrens,et al.  Neural Mechanisms of Foraging , 2012, Science.

[75]  David Badre,et al.  Working memory management and predicted utility , 2013, Front. Behav. Neurosci..

[76]  T. Robbins,et al.  Differential regulation of fronto-executive function by the monoamines and acetylcholine. , 2007, Cerebral cortex.

[77]  M. Botvinick,et al.  The intrinsic cost of cognitive control. , 2013, The Behavioral and brain sciences.

[78]  R. Dolan,et al.  Dopamine Enhances Model-Based over Model-Free Choice Behavior , 2012, Neuron.

[79]  P. Dayan,et al.  Opponency Revisited: Competition and Cooperation Between Dopamine and Serotonin , 2010, Neuropsychopharmacology.

[80]  Thomas E. Hazy,et al.  Banishing the homunculus: Making working memory work , 2006, Neuroscience.

[81]  Fabio Paglieri,et al.  The costs of delay: waiting versus postponing in intertemporal choice. , 2013, Journal of the experimental analysis of behavior.

[82]  John Duncan,et al.  Dynamic Construction of a Coherent Attentional State in a Prefrontal Cell Population , 2013, Neuron.

[83]  JaneR . Taylor,et al.  Supranormal Stimulation of D1 Dopamine Receptors in the Rodent Prefrontal Cortex Impairs Spatial Working Memory Performance , 1997, The Journal of Neuroscience.

[84]  T. Braver,et al.  What Is the Subjective Cost of Cognitive Effort? Load, Trait, and Aging Effects Revealed by Economic Preference , 2013, PloS one.

[85]  Karl J. Friston,et al.  Frontal, midbrain and striatal dopaminergic function in early and advanced Parkinson's disease A 3D [(18)F]dopa-PET study. , 1999, Brain : a journal of neurology.

[86]  G. Phillips,et al.  Enhanced acquisition of discriminative approach following intra-amygdala d-amphetamine , 1997, Psychopharmacology.

[87]  Joseph T. McGuire,et al.  Decision makers calibrate behavioral persistence on the basis of time-interval experience , 2012, Cognition.

[88]  A. Arnsten,et al.  Neuromodulation of Thought: Flexibilities and Vulnerabilities in Prefrontal Cortical Network Synapses , 2012, Neuron.

[89]  Jonathan D. Cohen,et al.  On the Control of Control: The Role of Dopamine in Regulating Prefrontal Function and Working Memory , 2007 .

[90]  R. Cools,et al.  Dopaminergic modulation of cognitive control: distinct roles for the prefrontal cortex and the basal ganglia. , 2010, Current pharmaceutical design.

[91]  Matthijs Baas,et al.  Working Memory Benefits Creative Insight, Musical Improvisation, and Original Ideation Through Maintained Task-Focused Attention , 2012, Personality & social psychology bulletin.

[92]  J. Fuster.  Prefrontal Cortex , 2018 .

[93]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[94]  Shinsuke Shimojo,et al.  Neural Computations Underlying Arbitration between Model-Based and Model-free Learning , 2013, Neuron.

[95]  Graham V. Williams,et al.  Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory , 2007, Nature Neuroscience.

[96]  J. Duncan.  An adaptive coding model of neural function in prefrontal cortex , 2001 .

[97]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[98]  Maarten A. S. Boksem,et al.  Mental fatigue: Costs and benefits , 2008, Brain Research Reviews.

[99]  Manfred G Kitzbichler,et al.  Cognitive Effort Drives Workspace Configuration of Human Brain Functional Networks , 2011, The Journal of Neuroscience.

[100]  A. Arnsten,et al.  Psychostimulants and motivated behavior: Arousal and cognition , 2013, Neuroscience & Biobehavioral Reviews.

[101]  Roshan Cools,et al.  Reward Acts on the pFC to Enhance Distractor Resistance of Working Memory Representations , 2014, Journal of Cognitive Neuroscience.

[102]  Torben Ott,et al.  Dopamine Receptors Differentially Enhance Rule Coding in Primate Prefrontal Cortex Neurons , 2014, Neuron.

[103]  A C Roberts,et al.  6-Hydroxydopamine lesions of the prefrontal cortex in monkeys enhance performance on an analog of the Wisconsin Card Sort Test: possible interactions with subcortical dopamine , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[104]  Stan B. Floresco,et al.  Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making , 2008, Cognitive, affective & behavioral neuroscience.

[105]  M. D’Esposito,et al.  Inverted-U–Shaped Dopamine Actions on Human Working Memory and Cognitive Control , 2011, Biological Psychiatry.

[106]  P. Redgrave,et al.  Is the short-latency dopamine response too short to signal reward error? , 1999, Trends in Neurosciences.

[107]  Matthew L. Dixon,et al.  The Decision to Engage Cognitive Control Is Driven by Expected Reward-Value: Neural and Behavioral Evidence , 2012, PloS one.

[108]  R. Cools,et al.  Striatal Dopamine and the Interface between Motivation and Cognition , 2011, Front. Psychology.

[109]  Y. Saalmann,et al.  The Pulvinar Regulates Information Transmission Between Cortical Areas Based on Attention Demands , 2012, Science.

[110]  R. Dolan,et al.  Dopamine, Time, and Impulsivity in Humans , 2010, The Journal of Neuroscience.

[111]  Joseph T. McGuire,et al.  Decision making and the avoidance of cognitive demand. , 2010, Journal of experimental psychology. General.

[112]  Menno Nijboer,et al.  Decision Making in Concurrent Multitasking: Do People Adapt to Task Interference? , 2013, PloS one.

[113]  W. Fias,et al.  Overlapping Neural Systems Represent Cognitive Effort and Reward Anticipation , 2014, PloS one.

[114]  J. Daunizeau,et al.  Neural Mechanisms Underlying Motivation of Mental Versus Physical Effort , 2012, PLoS biology.

[115]  Serotonin and dopamine differentially affect appetitive and aversive general Pavlovian-to-instrumental transfer , 2014, Psychopharmacology.

[116]  M. Botvinick,et al.  Motivation and cognitive control: from behavior to neural mechanism. , 2015, Annual review of psychology.

[117]  R. Dolan,et al.  Dopamine and Effort-Based Decision Making , 2011, Front. Neurosci..

[118]  M. D’Esposito,et al.  Dopaminergic modulation of distracter-resistance and prefrontal delay period signal , 2015, Psychopharmacology.

[119]  J. Salamone,et al.  The Mysterious Motivational Functions of Mesolimbic Dopamine , 2012, Neuron.

[120]  D. Zald,et al.  Dopaminergic Mechanisms of Individual Differences in Human Effort-Based Decision-Making , 2012, The Journal of Neuroscience.

[121]  Jonathan D. Cohen,et al.  The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function , 2013, Neuron.

[122]  C. Lustig,et al.  Deterministic functions of cortical acetylcholine , 2014, The European journal of neuroscience.

[123]  Judy Illes,et al.  Neurocognitive enhancement: what can we do and what should we do? , 2004, Nature Reviews Neuroscience.

[124]  M. D’Esposito,et al.  Working Memory Capacity Predicts Dopamine Synthesis Capacity in the Human Striatum , 2008, The Journal of Neuroscience.

[125]  T. Klingberg,et al.  Prefrontal cortex and basal ganglia control access to working memory , 2008, Nature Neuroscience.

[126]  B. Postle,et al.  The cognitive neuroscience of working memory. , 2007, Annual review of psychology.

[127]  Stan B. Floresco,et al.  Delay-dependent modulation of memory retrieval by infusion of a dopamine D1 agonist into the rat medial prefrontal cortex. , 2001 .

[128]  Christian C Ruff,et al.  Sensory processing: who's in (top‐down) control? , 2013, Annals of the New York Academy of Sciences.

[129]  T. Robbins,et al.  Chemistry of the adaptive mind , 2004, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[130]  J D Cohen,et al.  A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. , 1990, Science.