Sensitivity to value-driven attention is predicted by how we learn from value

Reward learning is known to influence the automatic capture of attention. This study examined how the rate of learning, after high- or low-value reward outcomes, can influence future transfers into value-driven attentional capture. Participants performed an instrumental learning task that was directly followed by an attentional capture task. A hierarchical Bayesian reinforcement model was used to infer individual differences in learning from high or low reward. Results showed a strong relationship between high-reward learning rates (or the weight that is put on learning after a high reward) and the magnitude of attentional capture with high-reward colors. Individual differences in learning from high or low rewards were further related to performance differences when high- or low-value distractors were present. These findings provide novel insight into the development of value-driven attentional capture by showing how information updating after desired or undesired outcomes can influence future deployments of automatic attention.

[1]  Michael X. Cohen,et al.  Dorsal Striatal–midbrain Connectivity in Humans Predicts How Reinforcements Are Used to Guide Decisions , 2009, Journal of Cognitive Neuroscience.

[2]  Brian A. Anderson,et al.  Value-driven attentional priority is context specific , 2014, Psychonomic Bulletin & Review.

[3]  E. Wagenmakers,et al.  Bayesian parameter estimation in the Expectancy Valence model of the Iowa gambling task , 2010 .

[4]  L. Chelazzi,et al.  Behavioral/systems/cognitive Reward Changes Salience in Human Vision via the Anterior Cingulate , 2022 .

[5]  Joshua W. Brown,et al.  Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex , 2005, Science.

[6]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  R. Desimone,et al.  Selective attention gates visual processing in the extrastriate cortex. , 1985, Science.

[9]  Fang Fang,et al.  Attention modulates neuronal correlates of interhemispheric integration and global motion perception. , 2014, Journal of vision.

[10]  Viola S. Störmer,et al.  Reward speeds up and increases consistency of visual selective attention: a lifespan comparison , 2014, Cognitive, affective & behavioral neuroscience.

[11]  Okihide Hikosaka,et al.  Reward-Dependent Gain and Bias of Visual Responses in Primate Superior Colliculus , 2003, Neuron.

[12]  Jacqueline Gottlieb,et al.  Attention, Reward, and Information Seeking , 2014, The Journal of Neuroscience.

[13]  R. K. Simpson.  Nature Neuroscience , 2022 .

[14]  T. Sejnowski,et al.  Neurocomputational models of working memory , 2000, Nature Neuroscience.

[15]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[16]  Anthony W. Sali,et al.  The role of reward prediction in the control of attention. , 2014, Journal of experimental psychology. Human perception and performance.

[17]  L. Chelazzi,et al.  Visual Selective Attention and the Effects of Monetary Rewards , 2006, Psychological science.

[18]  Helen Steingroever,et al.  Validating the PVL-Delta model for the Iowa gambling task , 2013, Front. Psychol..

[19]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[20]  J. Theeuwes.  Perceptual selectivity for color and form , 1992, Perception & psychophysics.

[21]  Jan Theeuwes,et al.  Exogenous visual orienting by reward. , 2014, Journal of vision.

[22]  Anne G E Collins,et al.  How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis , 2012, The European journal of neuroscience.

[23]  M. Peelen,et al.  Neural Mechanisms of Incentive Salience in Naturalistic Human Vision , 2015, Neuron.

[24]  Mark A. Straccia,et al.  Anterior Cingulate Engagement in a Foraging Context Reflects Choice Difficulty, Not Foraging Value , 2014, Nature Neuroscience.

[25]  References , 1971 .

[26]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[27]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[28]  Dean F. Wong,et al.  The Role of Dopamine in Value-Based Attentional Orienting , 2016, Current Biology.

[29]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[30]  M. Lee.  How cognitive modeling can benefit from hierarchical Bayesian models. , 2011 .

[31]  Jan Theeuwes,et al.  OpenSesame: An open-source, graphical experiment builder for the social sciences , 2011, Behavior Research Methods.

[32]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[33]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[34]  Patryk A. Laurent,et al.  Value-driven attentional capture , 2011, Proceedings of the National Academy of Sciences.

[35]  Nathaniel D. Daw,et al.  Trial-by-trial data analysis using computational models , 2011 .

[36]  J. Theeuwes,et al.  Top-down versus bottom-up attentional control: a failed theoretical dichotomy , 2012, Trends in Cognitive Sciences.

[37]  J. Serences,et al.  Value-based attentional capture influences context-dependent decision-making. , 2015, Journal of neurophysiology.

[38]  J. O'Doherty,et al.  The problem with value , 2014, Neuroscience & Biobehavioral Reviews.

[39]  Lourens J. Waldorp,et al.  Visual Information Shapes the Dynamics of Corticobasal Ganglia Pathways during Response Selection and Inhibition , 2015, Journal of Cognitive Neuroscience.

[40]  J. Theeuwes,et al.  Reward grabs the eye: Oculomotor capture by rewarding stimuli , 2012, Vision Research.

[41]  Timothy H. Muller,et al.  Reward Activates Stimulus-Specific and Task-Dependent Representations in Visual Association Cortices , 2014, The Journal of Neuroscience.

[42]  L. Chelazzi,et al.  Rewards teach visual selective attention , 2013, Vision Research.

[43]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[44]  Steven Yantis,et al.  Value-driven attentional and oculomotor capture during goal-directed, unconstrained viewing , 2012, Attention, Perception, & Psychophysics.

[45]  John-Dylan Haynes,et al.  Disentangling neural representations of value and salience in the human brain , 2014, Proceedings of the National Academy of Sciences.

[46]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[47]  Jon Driver,et al.  Reward Priority of Visual Target Singletons Modulates Event-Related Potential Signatures of Attentional Selection , 2009, Psychological science.

[48]  L. Chelazzi,et al.  Learning to Attend and to Ignore Is a Matter of Gains and Losses , 2009, Psychological science.

[49]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.