The control of tonic pain by active relief learning

Tonic pain after injury characterises a behavioural state that prioritises recovery. Although generally suppressing cognition and attention, tonic pain needs to allow effective relief learning to reduce the cause of the pain. Here, we describe a central learning circuit that supports learning of relief and concurrently suppresses the level of ongoing pain. We used computational modelling of behavioural, physiological and neuroimaging data in two experiments in which subjects learned to terminate tonic pain in static and dynamic escape-learning paradigms. In both studies, we show that active relief-seeking involves a reinforcement learning process manifest by error signals observed in the dorsal putamen. Critically, this system uses an uncertainty (‘associability’) signal detected in pregenual anterior cingulate cortex that both controls the relief learning rate, and endogenously and parametrically modulates the level of tonic pain. The results define a self-organising learning circuit that reduces ongoing pain when learning about potential relief.

[1]  Jane R. Garrison,et al.  Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies , 2013, Neuroscience & Biobehavioral Reviews.

[2]  B. Balleine,et al.  A specific role for posterior dorsolateral striatum in human habit learning , 2009, The European journal of neuroscience.

[3]  I. Kirsch,et al.  Is the rationale more important than deception? A randomized controlled trial of open-label placebo analgesia , 2017, Pain.

[4]  Karl J. Friston,et al.  Bayesian model selection for group studies — Revisited , 2014, NeuroImage.

[5]  V. Napadow,et al.  Disrupted functional connectivity of the periaqueductal gray in chronic low back pain , 2014, NeuroImage: Clinical.

[6]  Richard J. Davidson,et al.  Individual Differences in the Effects of Perceived Controllability on Pain Perception: Critical Role of the Prefrontal Cortex , 2007, Journal of Cognitive Neuroscience.

[7]  J. O'Doherty,et al.  Model‐Based fMRI and Its Application to Reward Learning and Decision Making , 2007, Annals of the New York Academy of Sciences.

[8]  R. Dolan,et al.  Uncertainty Increases Pain: Evidence for a Novel Mechanism of Pain Modulation Involving the Periaqueductal Gray , 2013, The Journal of Neuroscience.

[9]  T. Johnstone,et al.  Perceived Controllability Modulates the Neural Response to Pain , 2004, The Journal of Neuroscience.

[10]  K. Wiech,et al.  Dissociable Neural Mechanisms Underlying the Modulation of Pain and Anxiety? An fMRI Pilot Study , 2014, PloS one.

[11]  B. Vogt Pain and emotion interactions in subregions of the cingulate gyrus , 2005, Nature Reviews Neuroscience.

[12]  Ken-ichi Amemori,et al.  Localized Microstimulation of Primate Pregenual Cingulate Cortex Induces Negative Decision-Making , 2012, Nature Neuroscience.

[13]  C. Lévi-Strauss,et al.  Experimental investigation , 2013 .

[14]  Raymond J. Dolan,et al.  Striatal dysfunction during reversal learning in unmedicated schizophrenia patients☆ , 2014, NeuroImage.

[15]  P. Dayan,et al.  Serotonin Selectively Modulates Reward Value in Human Decision-Making , 2012, The Journal of Neuroscience.

[16]  J. Vlaeyen Learning to predict and control harmful events: chronic pain and conditioning , 2015, Pain.

[17]  Joseph W. Kable,et al.  Medial prefrontal cortical activity reflects dynamic re-evaluation during voluntary persistence , 2015, Nature Neuroscience.

[18]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[19]  Karl J. Friston,et al.  The Homeostatic Logic of Reward , 2018, bioRxiv.

[20]  Joseph E LeDoux,et al.  Avoiding Negative Outcomes: Tracking the Mechanisms of Avoidance Learning in Humans During Fear Conditioning , 2009, Front. Behav. Neurosci..

[21]  Peter C. Holland,et al.  Mini-review: Prediction errors, attention and associative learning , 2016, Neurobiology of Learning and Memory.

[22]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[23]  F. Benedetti,et al.  The role of learning in nocebo and placebo effects , 2008, PAIN.

[24]  E. Navratilova,et al.  Reward and motivation in pain and pain relief , 2014, Nature Neuroscience.

[25]  N. Daw,et al.  Differential roles of human striatum and amygdala in associative learning , 2011, Nature Neuroscience.

[26]  Daphna Shohamy,et al.  Representation of aversive prediction errors in the human periaqueductal gray , 2014, Nature Neuroscience.

[27]  Karl J. Friston,et al.  Bayesian model selection for group studies , 2009, NeuroImage.

[28]  R. Dolan,et al.  Computations of uncertainty mediate acute stress responses in humans , 2016, Nature Communications.

[29]  Brandon Galarita,et al.  Chronic , 2020, Definitions.

[30]  C. Büchel,et al.  Separate amygdala subregions signal surprise and predictiveness during associative fear learning in humans , 2013, The European journal of neuroscience.

[31]  D. Chialvo,et al.  Beyond Feeling: Chronic Pain Hurts the Brain, Disrupting the Default-Mode Network Dynamics , 2008, The Journal of Neuroscience.

[32]  P. Holland,et al.  Amount of training and stimulus salience affect associability changes in serial conditioning , 2002, Behavioural Processes.

[33]  T. Robbins,et al.  Dissociable Learning Processes Underlie Human Pain Conditioning , 2016, Current Biology.

[34]  J. Scholz,et al.  White matter integrity of the descending pain modulatory system is associated with interindividual differences in placebo analgesia , 2012, PAIN®.

[35]  Peter Bossaerts,et al.  Evidence for Model-based Computations in the Human Amygdala during Pavlovian Conditioning , 2013, PLoS Comput. Biol..

[36]  Nathaniel D. Daw,et al.  Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study , 2010, NeuroImage.

[37]  Karl J. Friston,et al.  Opponent appetitive-aversive neural processes underlie predictive learning of pain relief , 2005, Nature Neuroscience.

[38]  W. Krieg Functional Neuroanatomy , 1953, Springer Series in Experimental Entomology.

[39]  P. Schweinhardt,et al.  Doubling Your Payoff: Winning Pain Relief Engages Endogenous Pain Inhibition1,2,3 , 2015, eNeuro.

[40]  J. O'Doherty,et al.  Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain , 2006, PLoS biology.

[41]  Karl J. Friston,et al.  Action-Specific Value Signals in Reward-Related Regions of the Human Brain , 2012, The Journal of Neuroscience.

[42]  M. Pelley The Role of Associative History in Models of Associative Learning: A Selective Review and a Hybrid Model: , 2004 .

[43]  Richard J. Davidson,et al.  Functional neuroanatomy of aversion and its anticipation , 2006, NeuroImage.

[44]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[45]  J. H. Curtis,et al.  Learning Theory and Behavior , 1960 .

[46]  S. Clare,et al.  Imaging how attention modulates pain in humans using functional MRI. , 2002, Brain : a journal of neurology.

[47]  Karl J. Friston,et al.  A Bayesian Foundation for Individual Learning Under Uncertainty , 2011, Front. Hum. Neurosci..

[48]  Timothy E. J. Behrens,et al.  Neural Mechanisms of Foraging , 2012, Science.

[49]  Nikolaus Weiskopf,et al.  Anterolateral Prefrontal Cortex Mediates the Analgesic Effect of Expected and Perceived Control over Pain , 2006, The Journal of Neuroscience.

[50]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[51]  B. Vogt,et al.  Architecture and neurocytology of monkey cingulate gyrus , 2005, The Journal of comparative neurology.

[52]  S. Ochs Integrative Activity of the Brain: An Interdisciplinary Approach , 1968 .

[53]  D. Powell,et al.  Efferent connections of the medial prefrontal cortex in the rabbit , 1994, Experimental Brain Research.

[54]  H Szymańska,et al.  [Task performance]. , 1979, Pielegniarka i polozna.

[55]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[56]  I. Kirsch,et al.  Classical conditioning and the placebo effect , 1997, PAIN.

[57]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[58]  Lionel Rigoux,et al.  VBA: A Probabilistic Treatment of Nonlinear Models for Neurobiological and Behavioural Data , 2014, PLoS Comput. Biol..

[59]  R. Solomon,et al.  An opponent-process theory of motivation. I. Temporal dynamics of affect. , 1974, Psychological review.

[60]  Chantal Delon-Martin,et al.  Uncertainty in anticipation of uncomfortable rectal distension is modulated by the autonomic nervous system — A fMRI study in healthy volunteers , 2015, NeuroImage.

[61]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[62]  Till Sprenger,et al.  Distraction modulates connectivity of the cingulo-frontal cortex and the midbrain during pain—an fMRI analysis , 2004, Pain.

[63]  Thomas E. Nichols,et al.  Placebo Effects Mediated by Endogenous Opioid Activity on μ-Opioid Receptors , 2005, The Journal of Neuroscience.

[64]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[65]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[66]  Richard J. Davidson,et al.  Neural Emotion Regulation Circuitry Underlying Anxiolytic Effects of Perceived Control over Pain , 2014, Journal of Cognitive Neuroscience.

[67]  M. L. Le Pelley The Role of Associative History in Models of Associative Learning: A Selective Review and a Hybrid Model , 2004, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[68]  E. Vaadia,et al.  Midbrain dopamine neurons encode decisions for future action , 2006, Nature Neuroscience.

[69]  Anthony K. P. Jones,et al.  Pain processing during three levels of noxious stimulation produces differential patterns of central activity , 1997, Pain.

[70]  L. Vancleef,et al.  The interruptive effect of pain on attention. , 2006, The journal of pain : official journal of the American Pain Society.

[71]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[72]  C. Helmchen,et al.  Effects of perceived and exerted pain control on neural activity during pain relief in experimental heat hyperalgesia: A fMRI study , 2012, European journal of pain.

[73]  Dante R. Chialvo,et al.  Chronic pain patients are impaired on an emotional decision-making task , 2004, Pain.

[74]  Frank Baeyens,et al.  Habituation and the interference of pain with task performance , 1997, Pain.

[75]  C. Büchel,et al.  Mechanisms of placebo analgesia: rACC recruitment of a subcortical antinociceptive network , 2006, Pain.

[76]  B Bromm,et al.  Event-related potential correlates of interference between cognitive performance and tonic experimental pain. , 1997, Psychophysiology.

[77]  Peter Dayan,et al.  Temporal difference models describe higher-order learning in humans , 2004, Nature.

[78]  A. Dickinson Conditioning and associative learning. , 1981, British medical bulletin.

[79]  C. Büchel,et al.  Activation of the Opioidergic Descending Pain Control System Underlies Placebo Analgesia , 2009, Neuron.

[80]  Thomas J. Schnitzer,et al.  Corticostriatal functional connectivity predicts transition to chronic back pain , 2012, Nature Neuroscience.

[81]  K. Hegenscheid,et al.  Chronic Back Pain Is Associated With Decreased Prefrontal and Anterior Insular Gray Matter: Results From a Population-Based Cohort Study. , 2016, The journal of pain : official journal of the American Pain Society.

[82]  M. Baliki,et al.  Predicting Value of Pain and Analgesia: Nucleus Accumbens Response to Noxious Stimuli Changes in the Presence of Chronic Pain , 2010, Neuron.

[83]  Niels Birbaumer,et al.  The role of operant conditioning in chronic pain: an experimental investigation , 2002, Pain.

[84]  H. Watabe,et al.  Cerebral decreases in opioid receptor binding in patients with central neuropathic pain measured by [11C]diprenorphine binding and PET , 2004, European journal of pain.

[85]  V. B. Domesick Projections from the cingulate cortex in the rat. , 1969, Brain research.

[86]  M. Laslett Chronic back pain. , 1988 .