Unsigned value prediction-error modulates the motor system in absence of choice

Human actions are driven by the pursuit of goals, especially when achieving these goals entails a reward. Accordingly, recent work showed that anticipating a reward in a motor task influences the motor system, boosting motor excitability and increasing overall readiness. Attaining a reward typically requires some mental or physical effort. Recent neuroimaging evidence suggested that both reward expectation and effort requirements are encoded by a partially overlapping brain network. Moreover, reward and effort information are combined in an integrative value signal. However, whether and how mental effort is integrated with reward at the motor level during task preparation remains unclear. To address these issues, we implemented a mental effort task where reward expectation and effort requirements were manipulated. During task preparation, TMS was delivered on the motor cortex and motor-evoked potentials (MEPs) were recorded on the right hand muscles to probe motor excitability. The results showed an interaction of effort and reward in modulating the motor system, reflecting an unsigned value prediction-error signal. Crucially, this was observed in the motor system in absence of a value-based decision or value-driven action selection. This suggests a high-level cognitive factor such as unsigned value prediction-error can modulate the motor system. Interestingly, effort-related motor excitability was also modulated by individual differences in tendency to engage in (and enjoy) mental effort, as measured by the Need for Cognition questionnaire, underlining a role of subjective effort experience in value-driven preparation for action.

[1]  P. Sterling Allostasis: A model of predictive regulation , 2012, Physiology & Behavior.

[2]  Irwin P. Levin,et al.  Need for Cognition and Choice Framing Effects , 1996 .

[3]  C. Summerfield,et al.  Expectation (and attention) in visual cognition , 2009, Trends in Cognitive Sciences.

[4]  T. Prescott,et al.  The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. , 2010, Progress in Neurobiology.

[5]  C. S. E. Weston Another major function of the anterior cingulate cortex: The representation of requirements , 2012, Neuroscience & Biobehavioral Reviews.

[6]  Antonio Rangel,et al.  Neural computations associated with goal-directed choice , 2010, Current Opinion in Neurobiology.

[7]  Need for Cognition Related to Time Perception , 1986, Perceptual and motor skills.

[8]  C. F. Kao,et al.  The efficient assessment of need for cognition. , 1984, Journal of personality assessment.

[9]  S. White A dual role. , 2001 .

[10]  Mathias Pessiglione,et al.  Separate Valuation Subsystems for Delay and Effort Decision Costs , 2010, The Journal of Neuroscience.

[11]  R. Cools Dopaminergic control of the striatum for high-level cognition , 2011, Current Opinion in Neurobiology.

[12]  Karl J. Friston,et al.  Prediction, perception and agency , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  Bas Verplanken,et al.  Need for Cognition and External Information Search: Responses to Time Pressure during Decision-Making , 1993 .

[14]  Timothy E. J. Behrens,et al.  Review Frontal Cortex and Reward-guided Learning and Decision-making Figure 1. Frontal Brain Regions in the Macaque Involved in Reward-guided Learning and Decision-making Finer Grained Anatomical Divisions with Frontal Cortical Systems for Reward-guided Behavior , 2022 .

[15]  Colin Camerer,et al.  Transformation of stimulus value signals into motor commands during simple choice , 2011, Proceedings of the National Academy of Sciences.

[16]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[17]  J. Simpson DOPAMINERGIC MECHANISMS , 1976 .

[18]  Massimo Silvetti,et al.  Adaptive effort investment in cognitive and physical tasks: a neurocomputational model , 2015, Front. Behav. Neurosci..

[19]  Jin Fan,et al.  Common and distinct networks underlying reward valence and processing stages: A meta-analysis of functional neuroimaging studies , 2011, Neuroscience & Biobehavioral Reviews.

[20]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[21]  P. Cisek Integrated Neural Processes for Defining Potential Actions and Deciding between Them: A Computational Model , 2006, The Journal of Neuroscience.

[22]  A. Aron,et al.  Urges for food and money spill over into motor system excitability before action is taken , 2011, The European journal of neuroscience.

[23]  Jonathan D. Cohen,et al.  The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function , 2013, Neuron.

[24]  J. Cacioppo,et al.  DISPOSITIONAL DIFFERENCES IN COGNITIVE MOTIVATION : THE LIFE AND TIMES OF INDIVIDUALS VARYING IN NEED FOR COGNITION , 1996 .

[25]  Clay B. Holroyd,et al.  Motivation of extended behaviors by anterior cingulate cortex , 2012, Trends in Cognitive Sciences.

[26]  B Wedge,et al.  Conflict Management , 2019, Cases in Public Relations Management.

[27]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[28]  S. Bestmann,et al.  Time-Dependent Changes in Human Corticospinal Excitability Reveal Value-Based Competition for Action during Decision Processing , 2012, The Journal of Neuroscience.

[29]  André Vandierendonck,et al.  The role of working memory in carrying and borrowing , 2007, Psychological research.

[30]  N. Ramnani,et al.  The Anterior Cingulate Gyrus Signals the Net Value of Others' Rewards , 2014, The Journal of Neuroscience.

[31]  Dimitrios Kapogiannis,et al.  Reward‐related activity in the human motor cortex , 2008, The European journal of neuroscience.

[32]  A. Song,et al.  The involvement of the dopaminergic midbrain and cortico-striatal-thalamic circuits in the integration of reward prospect and attentional task demands. , 2012, Cerebral cortex.

[33]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[34]  K. R. Ridderinkhof,et al.  Emotional stimuli modulate readiness for action: A transcranial magnetic stimulation study , 2010, Cognitive, affective & behavioral neuroscience.

[35]  E. Olivier,et al.  Influence of Reward on Corticospinal Excitability during Movement Preparation , 2012, The Journal of Neuroscience.

[36]  S. White Dual role , 1996, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[37]  S. Dornič,et al.  Tolerance for mental effort: Self-ratings related to perception, performance and personality , 1991 .

[38]  T. Paus Primate anterior cingulate cortex: Where motor control, drive and cognition interface , 2001, Nature Reviews Neuroscience.

[39]  Jonathan D. Wallis,et al.  Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables , 2009, Journal of Cognitive Neuroscience.

[40]  M. Roesch,et al.  Attention for Learning Signals in Anterior Cingulate Cortex , 2011, The Journal of Neuroscience.

[41]  J. W. Aldridge,et al.  Dissecting components of reward: 'liking', 'wanting', and learning. , 2009, Current opinion in pharmacology.

[42]  Peter Dayan,et al.  Temporal difference models describe higher-order learning in humans , 2004, Nature.

[43]  T. Braver,et al.  What Is the Subjective Cost of Cognitive Effort? Load, Trait, and Aging Effects Revealed by Economic Preference , 2013, PloS one.

[44]  D. Zald,et al.  Dopaminergic Mechanisms of Individual Differences in Human Effort-Based Decision-Making , 2012, The Journal of Neuroscience.

[45]  T. Mima,et al.  Momentary reward induce changes in excitability of primary motor cortex , 2011, Clinical Neurophysiology.

[46]  D. Broadbent Perception and communication , 1958 .

[47]  Karl J. Friston,et al.  Predictive coding under the free-energy principle , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[48]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[49]  W. Fias,et al.  Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice , 2014, Neuropsychologia.

[50]  Timothy E. J. Behrens,et al.  Dissociable Reward and Timing Signals in Human Midbrain and Ventral Striatum , 2011, Neuron.

[51]  Jejo D. Koola,et al.  Emotion facilitates action: a transcranial magnetic stimulation study of motor cortex excitability during picture viewing. , 2007, Psychophysiology.

[52]  M. Rushworth,et al.  Behavioral / Systems / Cognitive Connectivity-Based Parcellation of Human Cingulate Cortex and Its Relation to Functional Specialization , 2008 .

[53]  Joseph T. McGuire,et al.  Decision making and the avoidance of cognitive demand. , 2010, Journal of experimental psychology. General.

[54]  W. Fias,et al.  Overlapping Neural Systems Represent Cognitive Effort and Reward Anticipation , 2014, PloS one.

[55]  Joshua W. Brown,et al.  From conflict management to reward-based decision making: Actors and critics in primate medial frontal cortex , 2014, Neuroscience & Biobehavioral Reviews.

[56]  Joshua W. Brown,et al.  Medial prefrontal cortex as an action-outcome predictor , 2011, Nature Neuroscience.

[57]  Ulrike Basten,et al.  How the brain integrates costs and bene fi ts during decision making , 2010 .

[58]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[59]  S. Sara,et al.  Orienting and Reorienting: The Locus Coeruleus Mediates Cognition through Arousal , 2012, Neuron.

[60]  R. Ratcliff,et al.  Bias in the Brain: A Diffusion Model Analysis of Prior Probability and Potential Payoff , 2012, The Journal of Neuroscience.

[61]  Karl J. Friston,et al.  A Dual Role for Prediction Error in Associative Learning , 2008, Cerebral cortex.

[62]  C. Carver,et al.  Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales , 1994 .

[63]  Karl J. Friston,et al.  Reflections on agranular architecture: predictive coding in the motor cortex , 2013, Trends in Neurosciences.

[64]  S. Sternberg Memory-scanning: mental processes revealed by reaction-time experiments. , 1969, American scientist.

[65]  Daeyeol Lee,et al.  Ubiquity and Specificity of Reinforcement Signals throughout the Human Brain , 2011, Neuron.

[66]  Joseph T. McGuire,et al.  Effort discounting in human nucleus accumbens , 2009, Cognitive, affective & behavioral neuroscience.

[67]  Massimo Silvetti,et al.  Value and Prediction Error in Medial Frontal Cortex: Integrating the Single-Unit and Systems Levels of Analysis , 2011, Front. Hum. Neurosci..

[68]  J. O'Doherty,et al.  Reward representations and reward-related learning in the human brain: insights from neuroimaging , 2004, Current Opinion in Neurobiology.

[69]  J. Kalaska,et al.  Neural mechanisms for interacting with a world full of action choices. , 2010, Annual review of neuroscience.

[70]  S. Haber,et al.  The Reward Circuit: Linking Primate Anatomy and Human Imaging , 2010, Neuropsychopharmacology.

[71]  Rolf Ulrich,et al.  Automatic and controlled stimulus processing in conflict tasks: Superimposed diffusion processes and delta functions , 2015, Cognitive Psychology.

[72]  Desmond J Oathes,et al.  Worry facilitates corticospinal motor response to transcranial magnetic stimulation , 2008, Depression and anxiety.