Associability-modulated loss learning is increased in posttraumatic stress disorder

Disproportionate reactions to unexpected stimuli in the environment are a cardinal symptom of posttraumatic stress disorder (PTSD). Here, we test whether these heightened responses are associated with disruptions in distinct components of reinforcement learning. Specifically, using functional neuroimaging, a loss-learning task, and a computational model-based approach, we assessed the mechanistic hypothesis that overreactions to stimuli in PTSD arise from anomalous gating of attention during learning (i.e., associability). Behavioral choices of combat-deployed veterans with and without PTSD were fit to a reinforcement learning model, generating trial-by-trial prediction errors (signaling unexpected outcomes) and associability values (signaling attention allocation to the unexpected outcomes). Neural substrates of associability value and behavioral parameter estimates of associability updating, but not prediction error, increased with PTSD during loss learning. Moreover, the interaction of PTSD severity with neural markers of associability value predicted behavioral choices. These results indicate that increased attention-based learning may underlie aspects of PTSD and suggest potential neuromechanistic treatment targets.

[1]  A. Taylor,et al.  Effects of varying scoring rules of the Clinician-Administered PTSD Scale (CAPS) for the diagnosis of post-traumatic stress disorder in motor vehicle accident victims. , 1995, Behaviour research and therapy.

[2]  Guillem R. Esber,et al.  Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain , 2012, The European journal of neuroscience.

[3]  B. Bradley,et al.  Fear Extinction in Traumatized Civilians with Posttraumatic Stress Disorder: Relation to Symptom Severity , 2011, Biological Psychiatry.

[4]  I. Liberzon,et al.  Biological studies of post-traumatic stress disorder , 2012, Nature Reviews Neuroscience.

[5]  Nathaniel D. Daw,et al.  Trial-by-trial data analysis using computational models , 2011 .

[6]  N. Daw,et al.  Rethinking Extinction , 2015, Neuron.

[7]  Guillem R. Esber,et al.  Attention-Related Pearce-Kaye-Hall Signals in Basolateral Amygdala Require the Midbrain Dopaminergic System , 2012, Biological Psychiatry.

[8]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[9]  Rany Abend,et al.  Effect of Attention Training on Attention Bias Variability and PTSD Symptoms: Randomized Controlled Trials in Israeli and U.S. Combat Veterans. , 2015, The American journal of psychiatry.

[10]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[11]  H. Hoeken,et al.  Evoking and Measuring Identification with Narrative Characters – A Linguistic Cues Framework , 2017, Front. Psychol..

[12]  N. Daw,et al.  Differential roles of human striatum and amygdala in associative learning , 2011, Nature Neuroscience.

[13]  B. Christensen,et al.  Measuring premorbid IQ in traumatic brain injury: An examination of the validity of the Wechsler Test of Adult Reading (WTAR) , 2008, Journal of clinical and experimental neuropsychology.

[14]  C. Mathys,et al.  Computational approaches to psychiatry , 2014, Current Opinion in Neurobiology.

[15]  M. Lund,et al.  The Combat Exposure Scale: a systematic assessment of trauma in the Vietnam War. , 1984, Journal of clinical psychology.

[16]  Guillem R. Esber,et al.  Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala , 2010, The Journal of Neuroscience.

[17]  Robert C. Wilson,et al.  Reinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms , 2015, The Journal of Neuroscience.

[18]  P. Schnurr,et al.  Cognitive behavioral therapy for posttraumatic stress disorder in women: a randomized controlled trial. , 2007, JAMA.

[19]  Guillem R. Esber,et al.  All that glitters ... dissociating attention and outcome expectancy from prediction errors signals. , 2010, Journal of neurophysiology.

[20]  Vanessa M Brown,et al.  Fear learning circuitry is biased toward generalization of fear associations in posttraumatic stress disorder , 2015, Translational Psychiatry.

[21]  N. Breslau,et al.  Intelligence and other predisposing factors in exposure to trauma and posttraumatic stress disorder: a follow-up study at age 17 years. , 2006, Archives of general psychiatry.

[22]  A. Gelman,et al.  Multiple Imputation with Diagnostics (mi) in R: Opening Windows into the Black Box , 2011 .

[23]  P. Dayan,et al.  Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis , 2013, Biology of Mood & Anxiety Disorders.

[24]  M. Lebreton,et al.  Assessing Inter-individual Variability in Brain-behavior Relationship Assessing Inter-individual Variability in Brain-behavior Relationship with Functional Neuroimaging , 2022 .

[25]  Colin Camerer,et al.  Thinking like a trader selectively reduces individuals' loss aversion , 2009, Proceedings of the National Academy of Sciences.

[26]  P. Montague,et al.  Activity in human ventral striatum locked to errors of reward prediction , 2002, Nature Neuroscience.

[27]  K. Amunts,et al.  Cytoarchitectonic mapping of the human amygdala, hippocampal region and entorhinal cortex: intersubject variability and probability maps , 2005, Anatomy and Embryology.

[28]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[29]  M. Craske,et al.  Maximizing exposure therapy: an inhibitory learning approach. , 2014, Behaviour research and therapy.

[30]  D. Pine,et al.  Cognitive control of attention is differentially affected in trauma-exposed individuals with and without post-traumatic stress disorder , 2012, Psychological Medicine.

[31]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[32]  Richie Poulton,et al.  Early childhood factors associated with the development of post-traumatic stress disorder: results from a longitudinal birth cohort , 2006, Psychological Medicine.

[33]  Margot J. Taylor,et al.  Soldiers With Posttraumatic Stress Disorder See a World Full of Threat: Magnetoencephalography Reveals Enhanced Tuning to Combat-Related Cues , 2015, Biological Psychiatry.

[34]  D. Pine,et al.  Biased emotional attention in post-traumatic stress disorder: a help as well as a hindrance? , 2007, Psychological Medicine.

[35]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[36]  R. Dolan,et al.  Computations of uncertainty mediate acute stress responses in humans , 2016, Nature Communications.

[37]  R A Steer,et al.  Dimensions of the Beck Depression Inventory-II in clinically depressed outpatients. , 1999, Journal of clinical psychology.

[38]  M. Pessiglione,et al.  Learning To Minimize Efforts versus Maximizing Rewards: Computational Principles and Neural Correlates , 2014, The Journal of Neuroscience.

[39]  M. Frank,et al.  Neural signature of hierarchically structured expectations predicts clustering and transfer of rule sets in reinforcement learning , 2016, Cognition.

[40]  Jane R. Garrison,et al.  Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies , 2013, Neuroscience & Biobehavioral Reviews.

[41]  Daniel S Pine,et al.  Threat-Related Attention Bias Variability and Posttraumatic Stress. , 2015, The American journal of psychiatry.

[42]  S. Lissek,et al.  Learning models of PTSD: Theoretical accounts and psychobiological evidence. , 2015, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[43]  D. Kaloupek,et al.  Predictors of Emotional Numbing, Revisited: A Replication and Extension , 2000, Journal of traumatic stress.

[44]  James F. Cavanagh,et al.  Cortical delta activity reflects reward prediction error and related behavioral adjustments, but at different times , 2015, NeuroImage.

[45]  Nicole Propst,et al.  Classical Conditioning Ii Current Research And Theory , 2016 .

[46]  Thomas V. Wiecki,et al.  Model-Based Cognitive Neuroscience Approaches to Computational Psychiatry , 2015 .

[47]  P. Holland,et al.  Amygdala circuitry in attentional and representational processes , 1999, Trends in Cognitive Sciences.

[48]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[49]  Timothy J. McDermott,et al.  Attention training normalises combat-related post-traumatic stress disorder effects on emotional Stroop performance using lexically matched word lists , 2016, Cognition & emotion.

[50]  Donna J. Calu,et al.  The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning , 2017, Front. Psychol..

[51]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[52]  Karl J. Friston,et al.  Computational psychiatry , 2012, Trends in Cognitive Sciences.

[53]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[54]  A. Etkin,et al.  Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. , 2007, The American journal of psychiatry.

[55]  Timothy Edward John Behrens,et al.  How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action , 2009, Neuron.

[56]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[57]  Tom A. B. Snijders,et al.  Multilevel Analysis , 2011, International Encyclopedia of Statistical Science.

[58]  M. Betancourt,et al.  Hamiltonian Monte Carlo for Hierarchical Models , 2013, 1312.0906.

[59]  Karl J. Friston,et al.  Bayesian model selection for group studies — Revisited , 2014, NeuroImage.

[60]  Florin Dolcos,et al.  The role of trauma-related distractors on neural systems for working memory and emotion processing in posttraumatic stress disorder. , 2009, Journal of psychiatric research.

[61]  D. Charney,et al.  The development of a Clinician-Administered PTSD Scale , 1995, Journal of traumatic stress.

[62]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 2005, IEEE Transactions on Neural Networks.

[63]  S. Rauch,et al.  Neurobiological Basis of Failure to Recall Extinction Memory in Posttraumatic Stress Disorder , 2009, Biological Psychiatry.

[64]  Lisa M. Shin,et al.  Emotion and cognition interactions in PTSD: a review of neurocognitive and neuroimaging studies , 2012, Front. Integr. Neurosci..

[65]  Anjali Krishnan,et al.  Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations , 2014, NeuroImage.

[66]  D. Dey,et al.  Current Trends in Bayesian Methodology with Applications , 2015 .

[67]  G. Dunbar,et al.  The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI , 1997, European Psychiatry.

[68]  Gregory McCarthy,et al.  Altered Resting-State Functional Connectivity of Basolateral and Centromedial Amygdala Complexes in Posttraumatic Stress Disorder , 2014, Neuropsychopharmacology.

[69]  M. O'Donnell,et al.  Posttraumatic stress disorder and depression following trauma: understanding comorbidity. , 2004, The American journal of psychiatry.

[70]  P. Dayan,et al.  Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning , 2017, Biological Psychiatry.

[71]  K. Ressler,et al.  An Overview of Translationally Informed Treatments for Posttraumatic Stress Disorder: Animal Models of Pavlovian Fear Conditioning to Human Clinical Trials , 2015, Biological Psychiatry.

[72]  M. Paulus,et al.  Executive function and PTSD: Disengaging from trauma , 2012, Neuropharmacology.

[73]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[74]  Jean-Philippe Langevin,et al.  Deep Brain Stimulation of the Basolateral Amygdala for Treatment-Refractory Posttraumatic Stress Disorder , 2016, Biological Psychiatry.

[75]  D. Schiller,et al.  What can fear and reward learning teach us about depression? , 2013, Current topics in behavioral neurosciences.

[76]  M. Pelley The Role of Associative History in Models of Associative Learning: A Selective Review and a Hybrid Model: , 2004 .

[77]  G. Glover,et al.  Dissociable Intrinsic Connectivity Networks for Salience Processing and Executive Control , 2007, The Journal of Neuroscience.

[78]  P. Schnurr,et al.  Cognitive Behavioral Therapy for Posttraumatic Stress Disorder in Women , 2017 .

[79]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[80]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[81]  M. Pessiglione,et al.  Critical Roles for Anterior Insula and Dorsal Striatum in Punishment-Based Avoidance Learning , 2012, Neuron.

[82]  R. Acierno,et al.  Prevalence Estimates of Combat-Related Post-Traumatic Stress Disorder: Critical Review , 2010, The Australian and New Zealand journal of psychiatry.

[83]  T. Robbins,et al.  Decision Making, Affect, and Learning: Attention and Performance XXIII , 2011 .

[84]  A. Zinchenko,et al.  Content specificity of attentional bias to threat in post-traumatic stress disorder. , 2017, Journal of anxiety disorders.

[85]  M. Frank,et al.  From reinforcement learning models to psychiatric and neurological disorders , 2011, Nature Neuroscience.

[86]  Y. Bar-Haim,et al.  Threat-related attentional bias in anxious and nonanxious individuals: a meta-analytic study. , 2007, Psychological bulletin.

[87]  Lisa M. Shin,et al.  Neurocircuitry Models of Posttraumatic Stress Disorder and Extinction: Human Neuroimaging Research—Past, Present, and Future , 2006, Biological Psychiatry.

[88]  C. Büchel,et al.  Separate amygdala subregions signal surprise and predictiveness during associative fear learning in humans , 2013, The European journal of neuroscience.