The exploration-exploitation dilemma in pain: an experimental investigation.

ABSTRACT Daily life consists of a chain of decisions. Typically, individuals may choose to pursue what they already know (exploitation), or to search for other options (exploration). This exploration-exploitation dilemma is a topic of interest across multiple scientific fields. Here we propose that investigating how individuals solve this dilemma may improve our understanding of how individuals make behavioral decisions (e.g., avoidance) when facing pain. To this end, we present the data of three experiments in which healthy individuals were given the opportunity to choose between four different movements, with each movement being associated with different probabilities of receiving a painful outcome only (Experiment 1), or pain and/or a reward (Experiment 2). We also investigated whether participants stuck to their decisions when the contingencies between each movement and the painful/rewarding outcome changed during the task (Experiment 3). The key findings across all experiments are the following: First, after initial exploration, participants most often exploited the safest option. Second, participants weighted rewards more heavily than receiving pain. Lastly, after receiving a painful outcome, participants were more inclined to explore than to exploit a rewarding movement. We argue that by focusing more on how individuals in pain solve the exploration-exploitation dilemma is helpful in understanding behavioral decision-making in pain.

[1]  Angelos-Miltiadis Krypotos,et al.  Decomposing conditioned avoidance performance with computational models. , 2020, Behaviour research and therapy.

[2]  J. Vlaeyen,et al.  Avoid or engage? Outcomes of graded exposure in youth with chronic pain using a sequential replicated single-case randomized design. , 2019, Pain.

[3]  I. Klugkist,et al.  A step-by-step guide on preregistration and effective data sharing for psychopathology research. , 2019, Journal of abnormal psychology.

[4]  Neil Garrett,et al.  Biased belief updating and suboptimal choice in foraging decisions , 2019, bioRxiv.

[5]  Rebecca L. Bond,et al.  Altered learning under uncertainty in unmedicated mood and anxiety disorders , 2019, Nature Human Behaviour.

[6]  I. Engelhard,et al.  A systematic review and meta-analysis of the evidence for unaware fear conditioning , 2019, Neuroscience & Biobehavioral Reviews.

[7]  A. Meulders From fear of movement-related pain and avoidance to chronic pain disability: a state-of-the-art review , 2019, Current Opinion in Behavioral Sciences.

[8]  I. Engelhard,et al.  Targeting avoidance via compound extinction , 2019, Cognition & emotion.

[9]  H. Flor,et al.  Contingency awareness as a prerequisite for differential contextual fear conditioning , 2018, Cognitive, Affective, & Behavioral Neuroscience.

[10]  B. Vervliet,et al.  The validity of human avoidance paradigms. , 2018, Behaviour research and therapy.

[11]  J. Vlaeyen,et al.  Exposure and CBT for Chronic Back Pain: An RCT on Differential Efficacy and Optimal Length of Treatment , 2018, Journal of consulting and clinical psychology.

[12]  B. Hommel,et al.  Motivational sensitivity of outcome-response priming: Experimental research and theoretical models , 2018, Psychonomic Bulletin & Review.

[13]  I. Klugkist,et al.  Bayesian hypothesis testing for human threat conditioning research: an introduction and the condir R package , 2017, European journal of psychotraumatology.

[14]  T. Beckers,et al.  A Primer on Bayesian Analysis for Experimental Psychopathologists , 2017, Journal of experimental psychopathology.

[15]  P. Onghena,et al.  Expose or protect? A randomized controlled trial of exposure in vivo vs pain-contingent treatment as usual in patients with complex regional pain syndrome type 1 , 2016, Pain.

[16]  G. Crombez,et al.  Pain in context: Cues predicting a reward decrease fear of movement related pain and avoidance behavior. , 2016, Behaviour research and therapy.

[17]  G. Crombez,et al.  The fear-avoidance model of pain. , 2016, Pain.

[18]  Lei Zhang,et al.  Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package , 2016, bioRxiv.

[19]  Mathijs Franssen,et al.  Acquisition and extinction of operant pain-related avoidance behavior using a 3 degrees-of-freedom robotic arm , 2016, Pain.

[20]  Stephan Hamann,et al.  Individual differences in sensitivity to reward and punishment and neural activity during reward and avoidance learning. , 2015, Social cognitive and affective neuroscience.

[21]  Geert Crombez,et al.  Pain-avoidance versus reward-seeking: an experimental investigation , 2015, Pain.

[22]  Aki Vehtari,et al.  Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC , 2015, Statistics and Computing.

[23]  M. Frank,et al.  Biases in the Explore–Exploit Tradeoff in Addictions: The Role of Avoidance of Uncertainty , 2015, Neuropsychopharmacology.

[24]  Robert C. Wilson,et al.  Reinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms , 2015, The Journal of Neuroscience.

[25]  Daphna Shohamy,et al.  Representation of aversive prediction errors in the human periaqueductal gray , 2014, Nature Neuroscience.

[26]  M. Lee,et al.  Bayesian Cognitive Modeling: A Practical Course , 2014 .

[27]  Johan W S Vlaeyen,et al.  Confirmatory factor analysis of the Dutch Intolerance of Uncertainty Scale: Comparison of the full and short version. , 2013, Journal of behavior therapy and experimental psychiatry.

[28]  J. Vlaeyen,et al.  The acquisition and generalization of cued and contextual pain-related fear: An experimental study using a voluntary movement paradigm , 2013, PAIN.

[29]  G. Crombez,et al.  Fear-Avoidance Model of Chronic Pain: The Next Generation , 2012, The Clinical journal of pain.

[30]  Karl J. Friston,et al.  Computational psychiatry , 2012, Trends in Cognitive Sciences.

[31]  J. Vlaeyen,et al.  The acquisition of fear of movement-related pain and associative learning: A novel pain-relevant human fear conditioning paradigm , 2011, PAIN.

[32]  E. Carpino,et al.  The Fear of Pain Questionnaire (FOPQ): assessment of pain-related fear among children and adolescents with chronic pain. , 2011, The journal of pain : official journal of the American Pain Society.

[33]  Z. Dienes Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[34]  Marcel A. van den Hout,et al.  Neuroticism and avoidance of ambiguous stimuli: Better safe than sorry? , 2010 .

[35]  M. Roy Weighting Pain Avoidance and Reward Seeking: A Neuroeconomical Approach to Pain , 2010, The Journal of Neuroscience.

[36]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[37]  J. Hoogstraten,et al.  Dutch translation of the Fear of Pain Questionnaire: Factor structure, reliability and validity , 2006, European journal of pain.

[38]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[39]  S. Sanders Behavioral conceptualization and treatment for chronic pain. , 2006 .

[40]  P. Muris,et al.  Psychometric Properties of the Dutch BIS/BAS Scales , 2005 .

[41]  K. Buhr,et al.  The Intolerance of Uncertainty Scale: psychometric properties of the English version. , 2002, Behaviour research and therapy.

[42]  J. Vlaeyen,et al.  The effects of failure feedback and pain-related fear on pain report, pain tolerance, and pain avoidance in chronic low back pain patients , 2001, Pain.

[43]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[44]  Michael N. Katehakis,et al.  The Multi-Armed Bandit Problem: Decomposition and Computation , 1987, Math. Oper. Res..

[45]  Richard D. Morey,et al.  Baysefactor: Computation of Bayes Factors for Common Designs , 2018 .

[46]  Ben R. Newell,et al.  Unpacking the Exploration–Exploitation Tradeoff: A Synthesis of Human and Animal Literatures , 2015 .

[47]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[48]  G. Crombez,et al.  Short running title : COMPETING GOALS ATTENUATE AVOIDANCE BEHAVIOR 1 COMPETING GOALS ATTENUATE AVOIDANCE BEHAVIOR IN THE CONTEXT OF PAIN , 2014 .

[49]  John K Kruschke,et al.  Bayesian Assessment of Null Values Via Parameter Estimation and Model Comparison , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[50]  Michael Moutoussis,et al.  Are computational models of any use to psychiatry? , 2011, Neural Networks.

[51]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[52]  H. Eysenck,et al.  Manual of the Eysenck personality questionnaire , 1975 .