Punishment-Induced Behavioral and Neurophysiological Variability Reveals Dopamine-Dependent Selection of Kinematic Movement Parameters

Action selection describes the high-level process that selects between competing movements. In animals, behavioral variability is critical for the motor exploration required to select the action that optimizes reward and minimizes cost/punishment and is guided by dopamine (DA). The aim of this study was to test in humans whether low-level movement parameters are affected by punishment and reward in ways similar to high-level action selection. Moreover, we addressed the proposed dependence of behavioral and neurophysiological variability on DA and whether this may underpin the exploration of kinematic parameters. Participants performed an out-and-back index finger movement and were instructed that monetary reward and punishment were based on its maximal acceleration (MA). In fact, the feedback was not contingent on the participant's behavior but predetermined. Blocks highly biased toward punishment were associated with increased MA variability relative to blocks either with reward or without feedback. This increase in behavioral variability was positively correlated with neurophysiological variability, as measured by changes in corticospinal excitability with transcranial magnetic stimulation over the primary motor cortex. Following the administration of a DA antagonist, the variability associated with punishment diminished and the correlation between behavioral and neurophysiological variability no longer existed. Similar changes in variability were not observed when participants executed a predetermined MA, nor did DA influence resting neurophysiological variability. Thus, under conditions of punishment, DA-dependent processes influence the selection of low-level movement parameters. We propose that the enhanced behavioral variability reflects the exploration of kinematic parameters for less punishing, or conversely more rewarding, outcomes.

[1]  O. Hikosaka,et al.  Modulation of saccadic eye movements by predicted reward outcome , 2001, Experimental Brain Research.

[2]  Michael J. Frank,et al.  A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. , 2006, Behavioral neuroscience.

[3]  M. Nicolelis,et al.  Differential Corticostriatal Plasticity during Fast and Slow Motor Skill Learning in Mice , 2004, Current Biology.

[4]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[5]  J. Kalaska,et al.  Neural mechanisms for interacting with a world full of action choices. , 2010, Annual review of neuroscience.

[6]  Anatol C. Kreitzer,et al.  Distinct roles for direct and indirect pathway striatal neurons in reinforcement , 2012, Nature Neuroscience.

[7]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[8]  M. C. Newland,et al.  Reinforcing behavioral variability: An analysis of dopamine-receptor subtypes and intermittent reinforcement , 2011, Pharmacology, Biochemistry and Behavior.

[9]  M. Nitsche,et al.  Dopaminergic modulation of long-lasting direct current-induced cortical excitability changes in the human motor cortex. , 2006, The European journal of neuroscience.

[10]  Reza Shadmehr,et al.  Learning from Sensory and Reward Prediction Errors during Motor Adaptation , 2011, PLoS Comput. Biol..

[11]  D. Perkel,et al.  Striatal Dopamine Modulates Basal Ganglia Output and Regulates Social Context-Dependent Behavioral Variability through D1 Receptors , 2010, The Journal of Neuroscience.

[12]  W. Schultz.  Multiple dopamine functions at different time courses. , 2007, Annual review of neuroscience.

[13]  M. Frank,et al.  Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration , 2009, The Journal of Neuroscience.

[14]  Sachie K. Ogawa,et al.  Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons , 2012, Neuron.

[15]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.

[16]  R. Costa.  A selectionist account of de novo action learning , 2011, Current Opinion in Neurobiology.

[17]  Ryan D Ward,et al.  Effects of D-amphetamine and ethanol on variable and repetitive key-peck sequences in pigeons. , 2006, Journal of the experimental analysis of behavior.

[18]  Rui M. Costa,et al.  Rapid Alterations in Corticostriatal Ensemble Coordination during Acute Dopamine-Dependent Motor Dysfunction , 2006, Neuron.

[19]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[20]  Binith Cheeran,et al.  Human θ burst stimulation enhances subsequent motor learning and increases performance variability. , 2011, Cerebral cortex.

[21]  Karl J. Friston,et al.  Influence of Uncertainty and Surprise on Human Corticospinal Excitability during Preparation for Action , 2008, Current Biology.

[22]  J. Krakauer,et al.  Why Don't We Move Faster? Parkinson's Disease, Movement Vigor, and Implicit Motivation , 2007, The Journal of Neuroscience.

[23]  P. Celnik,et al.  Modulation of Cerebellar Excitability by Polarity-Specific Noninvasive Direct Current Stimulation , 2009, The Journal of Neuroscience.

[24]  R. Dolan,et al.  How the Brain Translates Money into Force: A Neuroimaging Study of Subliminal Motivation , 2007, Science.

[25]  D. Perkel,et al.  Striatal dopamine modulates song spectral but not temporal features through D1 receptors , 2012, The European journal of neuroscience.

[26]  D. Deleu,et al.  Clinical Pharmacokinetic and Pharmacodynamic Properties of Drugs Used in the Treatment of Parkinson’s Disease , 2002, Clinical pharmacokinetics.

[27]  M. Fee,et al.  A hypothesis for basal ganglia-dependent reinforcement learning in the songbird , 2011, Neuroscience.

[28]  J. Krakauer,et al.  Are We Ready for a Natural History of Motor Learning? , 2011, Neuron.

[29]  J. Rothwell,et al.  D2 Receptor Block Abolishes Theta Burst Stimulation-Induced Neuroplasticity in the Human Motor Cortex , 2011, Neuropsychopharmacology.

[30]  W. Schultz.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.