Timing along the cardiac cycle modulates neural signals of reward-based learning

Natural fluctuations in cardiac activity influence brain activity associated with sensory stimuli and affect perceptual decisions about low magnitude, near-threshold stimuli. However, little is known about the impact of fluctuations in heart activity on other internal representations. Here we investigate cardiac influences on learning-related internal representations – absolute and signed prediction errors. By combining machine learning techniques with electroencephalography (EEG) and both simple, direct indices of task performance and computational model-derived indices of learning, we demonstrate that just as people are more sensitive to low magnitude, near threshold sensory stimuli in certain cardiac phases, so are they more sensitive to low magnitude absolute prediction errors in the same cycles. Importantly, however, this occurs even when the low magnitude prediction errors are associated with clearly suprathreshold sensory events. In addition, participants exhibiting stronger difference in their prediction errors representations between cardiac cycles exhibited higher learning rates and greater task accuracy.

[1]  Goal-directed recruitment of Pavlovian biases through selective visual attention , 2022 .

[2]  J. Kilner,et al.  Active tactile discrimination is coupled with and modulated by the cardiac cycle , 2022, bioRxiv.

[3]  B. Averbeck,et al.  Reinforcement-learning in fronto-striatal circuits , 2021, Neuropsychopharmacology.

[4]  R. Cools,et al.  Striatal BOLD and midfrontal theta power express motivation for action , 2020, bioRxiv.

[5]  M. Roesch,et al.  Anterior Cingulate Cortex Signals Attention in a Social Paradigm that Manipulates Reward and Shock , 2020, Current Biology.

[6]  Bolton K. H. Chau,et al.  Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys , 2020, Nature Communications.

[7]  James M. Kilner,et al.  Active sampling in visual search is coupled to the cardiac cycle , 2018, Cognition.

[8]  A. Villringer,et al.  Heart–brain interactions shape somatosensory perception and evoked potentials , 2019, Proceedings of the National Academy of Sciences.

[9]  M. Philiastides,et al.  Neural correlates of weighted reward prediction error during reinforcement learning classify response to cognitive behavioral therapy in depression , 2019, Science Advances.

[10]  A. Villringer,et al.  Interactions between cardiac activity and conscious somatosensory perception , 2019, bioRxiv.

[11]  M. Philiastides,et al.  Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta‐analysis , 2018, Human brain mapping.

[12]  Bolton K. H. Chau,et al.  The macaque anterior cingulate cortex translates counterfactual choice value into actual behavioral change , 2018, Nature Neuroscience.

[13]  Y. Niv,et al.  Model-based predictions for dopamine , 2018, Current Opinion in Neurobiology.

[14]  H. Critchley,et al.  The influence of physiological signals on cognition , 2018, Current Opinion in Behavioral Sciences.

[15]  Karen J. Mullinger,et al.  Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans , 2017, Scientific Reports.

[16]  Yuan Chang Leong,et al.  Dynamic Interaction between Reinforcement Learning and Attention in Multidimensional Environments , 2017, Neuron.

[17]  Craig G. Richter,et al.  Neural Responses to Heartbeats in the Default Network Encode the Self in Spontaneous Thoughts , 2016, The Journal of Neuroscience.

[18]  Wolfram Schultz,et al.  Dopamine reward prediction-error signalling: a two-component response , 2016, Nature Reviews Neuroscience.

[19]  Joshua W. Brown,et al.  Neural Mechanisms of Credit Assignment in a Multicue Environment , 2016, The Journal of Neuroscience.

[20]  J. Bhattacharya,et al.  Aroused with heart: Modulation of heartbeat evoked potential by arousal induction and its oscillatory correlates , 2015, Scientific Reports.

[21]  Charles Wyplosz The common currency , 2015 .

[22]  M. Philiastides,et al.  TITLE : Two spatiotemporally distinct value systems shape reward-based learning in the human brain , 2015 .

[23]  W. K. Simmons,et al.  Interoceptive predictions in the brain , 2015, Nature Reviews Neuroscience.

[24]  Robert C. Wilson,et al.  Reinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms , 2015, The Journal of Neuroscience.

[25]  Hong-Hsiang Liu,et al.  Effects of affective arousal on choice behavior, reward prediction errors, and feedback-related negativities in human reward-based decision making , 2015, Front. Psychol..

[26]  Marios G. Philiastides,et al.  Neural representations of confidence emerge from the process of decision formation during perceptual choices , 2015, NeuroImage.

[27]  C. Tallon-Baudry,et al.  Spontaneous fluctuations in neural responses to heartbeats predict visual detection , 2014, Nature Neuroscience.

[28]  K. Sakai,et al.  Autonomous Mechanism of Internal Choice Estimate Underlies Decision Inertia , 2014, Neuron.

[29]  N. Logothetis,et al.  Modular architectonic organization of the insula in the macaque monkey , 2014, The Journal of comparative neurology.

[30]  Andreas Schulze-Bonhage,et al.  Heart cycle-related effects on event-related potentials, spectral power changes, and connectivity patterns in the human ECoG , 2013, NeuroImage.

[31]  Joseph W. Kable,et al.  The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value , 2013, NeuroImage.

[32]  Andy Clark,et al.  The many faces of precision (Replies to commentaries on “Whatever next? Neural prediction, situated agents, and the future of cognitive science”) , 2013, Front. Psychol..

[33]  P. Tobler,et al.  Salience Signals in the Right Temporoparietal Junction Facilitate Value-Based Decisions , 2013, The Journal of Neuroscience.

[34]  S. Debener,et al.  Heartbeat evoked potentials mirror altered body perception in depressed patients , 2012, Clinical Neurophysiology.

[35]  H. Critchley,et al.  Emotional appraisal is influenced by cardiac afferent information. , 2012, Emotion.

[36]  John M. Pearson,et al.  Neuronal basis of sequential foraging decisions in a patchy environment , 2011, Nature Neuroscience.

[37]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[38]  M. Paulus,et al.  The role of interoception and alliesthesia in addiction , 2009, Pharmacology Biochemistry and Behavior.

[39]  Hugo D. Critchley,et al.  Following One's Heart: Cardiac Rhythms Gate Central Initiation of Sympathetic Reflexes , 2009, The Journal of Neuroscience.

[40]  P. Dayan,et al.  Decision theory, reinforcement learning, and the brain , 2008, Cognitive, affective & behavioral neuroscience.

[41]  K. Saleem,et al.  Complementary circuits connecting the orbital and medial prefrontal networks with the temporal, insular, and opercular cortex in the macaque monkey , 2008, The Journal of comparative neurology.

[42]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[43]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[44]  David McIntyre,et al.  Psychomotor speed in hypertension: effects of reaction time components, stimulus modality, and phase of the cardiac cycle. , 2007, Psychophysiology.

[45]  S. Sutton Predicting and Explaining Intentions and Behavior: How Well Are We Doing? , 1998 .

[46]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[47]  C. Sandman,et al.  Augmentation of the auditory event related potentials of the brain during diastole. , 1984, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.