The attention habit: how reward learning shapes attentional selection

There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task‐specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal‐directed (endogenous) versus stimulus‐driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward‐associated stimuli can automatically capture attention even when physically inconspicuous and task‐irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up‐to‐date account of value‐driven attention and some of its broader implications.

[1]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[2]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[3]  S. Yantis,et al.  Abrupt visual onsets and selective attention: evidence from visual search. , 1984, Journal of experimental psychology. Human perception and performance.

[4]  Susan L. Franzel,et al.  Guided search: an alternative to the feature integration model for visual search. , 1989, Journal of experimental psychology. Human perception and performance.

[5]  J. C. Johnston,et al.  On the locus of visual selection: evidence from focused attention tasks. , 1990, Journal of experimental psychology. Human perception and performance.

[6]  J. C. Johnston,et al.  Involuntary covert orienting is contingent on attentional control settings. , 1992, Journal of experimental psychology. Human perception and performance.

[7]  J. Theeuwes Perceptual selectivity for color and form , 1992, Perception & psychophysics.

[8]  K. Berridge,et al.  The neural basis of drug craving: An incentive-sensitization theory of addiction , 1993, Brain Research Reviews.

[9]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[10]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[11]  H. Egeth,et al.  Overriding stimulus-driven attentional capture , 1994, Perception & psychophysics.

[12]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[13]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[14]  S. Yantis,et al.  Visual attention: control, representation, and time course. , 1997, Annual review of psychology.

[15]  K. Hugdahl,et al.  Selective processing of visual alcohol cues in abstinent alcoholics: an approach-avoidance conflict? , 1997, Addictive behaviors.

[16]  M. Goldberg,et al.  The representation of visual salience in monkey parietal cortex , 1998, Nature.

[17]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[18]  M. Chun,et al.  Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention , 1998, Cognitive Psychology.

[19]  J. Deakin,et al.  Attentional bias for drug cues in opiate dependence , 2000, Psychological Medicine.

[20]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[21]  W. Schneider,et al.  Automatic attraction of attention to former targets in visual displays of letters , 2001, Perception & psychophysics.

[22]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[23]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[24]  Matt Field,et al.  Eye movements to smoking-related pictures in smokers: relationship between attentional biases and implicit and explicit measures of stimulus valence. , 2003, Addiction.

[25]  S. Yantis,et al.  Visual Attention: Bottom-Up Versus Top-Down , 2004, Current Biology.

[26]  G. Crombez,et al.  Does imminent threat capture and hold attention? , 2004, Emotion.

[27]  J. Maunsell Neuronal representations of cognitive state: reward or attention? , 2004, Trends in Cognitive Sciences.

[28]  C. Frith,et al.  Neural Correlates of Attentional Capture in Visual Search , 2004, Journal of Cognitive Neuroscience.

[29]  Andrew B. Leber,et al.  Coordination of Voluntary and Stimulus-Driven Attentional Control in Human Cortex , 2005, Psychological science.

[30]  Stephen D. Smith,et al.  An emotion-induced attentional blink elicited by aversively conditioned stimuli. , 2006, Emotion.

[31]  Monique Ernst,et al.  Increased Occupancy of Dopamine Receptors in Human Striatum during Cue-Elicited Cocaine Craving , 2006, Neuropsychopharmacology.

[32]  Jacqueline Gottlieb,et al.  Integration of Exogenous Input into a Dynamic Salience Map Revealed by Perturbing Attention , 2006, The Journal of Neuroscience.

[33]  N. Volkow,et al.  Cocaine Cues and Dopamine in Dorsal Striatum: Mechanism of Craving in Cocaine Addiction , 2006, The Journal of Neuroscience.

[34]  K. Affeld,et al.  Injection of granular material , 2006, J. Vis..

[35]  Mark F Bear,et al.  Reward timing in the primary visual cortex. , 2006, Science.

[36]  Andrew B. Leber,et al.  It’s under control: Top-down search strategies can override attentional capture , 2006, Psychonomic bulletin & review.

[37]  L. Chelazzi,et al.  Visual Selective Attention and the Effects of Monetary Rewards , 2006, Psychological science.

[38]  W. van den Brink,et al.  Attentional bias predicts heroin relapse following treatment. , 2006, Addiction.

[39]  L. Pessoa,et al.  Motivation sharpens exogenous spatial attention. , 2007, Emotion.

[40]  S. Yantis,et al.  Spatially selective representations of voluntary and stimulus-driven attentional priority in human occipital, parietal, and frontal cortex. , 2007, Cerebral cortex.

[41]  M. Field,et al.  Attentional bias in addictive behaviors: a review of its development, causes, and consequences. , 2008, Drug and alcohol dependence.

[42]  A. Graybiel Habits, rituals, and the evaluative brain. , 2008, Annual review of neuroscience.

[43]  Martin Eimer,et al.  Involuntary Attentional Capture is Determined by Task Set: Evidence from Event-related Brain Potentials , 2008, Journal of Cognitive Neuroscience.

[44]  John T Serences,et al.  Value-Based Modulations in Human Visual Cortex , 2008, Neuron.

[45]  L. Chelazzi,et al.  Learning to Attend and to Ignore Is a Matter of Gains and Losses , 2009, Psychological science.

[46]  Jon Driver,et al.  Reward Priority of Visual Target Singletons Modulates Event-Related Potential Signatures of Attentional Selection , 2009, Psychological science.

[47]  Pietro Perona,et al.  Homo economicus in visual search. , 2009, Journal of vision.

[48]  Aaron R. Seitz,et al.  Rewards Evoke Learning of Unconsciously Processed Visual Stimuli in Adult Humans , 2009, Neuron.

[49]  Christopher J. Peck,et al.  Reward Modulates Attention Independently of Action Value in Posterior Parietal Cortex , 2009, The Journal of Neuroscience.

[50]  Jia Liu,et al.  Dissociation of attention and intention in human posterior parietal cortex: an fMRI study , 2009, The European journal of neuroscience.

[51]  G. Campana,et al.  Where perception meets memory: A review of repetition priming in visual search tasks , 2010, Attention, perception & psychophysics.

[52]  C. Folk,et al.  Target-uncertainty effects in attentional capture: Color-singleton set or multiple attentional control settings? , 2010, Psychonomic bulletin & review.

[53]  Jan Theeuwes,et al.  Reward Guides Vision when It's Your Thing: Trait Reward-Seeking in Reward-Mediated Visual Priming , 2010, PloS one.

[54]  L. Pessoa,et al.  Embedding Reward Signals into Perception and Cognition , 2010, Front. Neurosci..

[55]  John T Serences,et al.  Population response profiles in early visual cortex are biased in favor of more valuable stimuli. , 2010, Journal of neurophysiology.

[56]  Satoshi Someya,et al.  An experimental investigation of flow-induced acoustic resonance and flow field in a closed side branch system using a high time-resolved PIV technique , 2010, J. Vis..

[57]  C. N. Boehler,et al.  The influence of reward associations on conflict processing in the Stroop task , 2010, Cognition.

[58]  J. Theeuwes Top-down and bottom-up control of visual selection. , 2010, Acta psychologica.

[59]  L. Chelazzi,et al.  Behavioral/systems/cognitive Reward Changes Salience in Human Vision via the Anterior Cingulate , 2022 .

[60]  Jan Theeuwes,et al.  What is top-down about contingent capture? , 2010, Attention, perception & psychophysics.

[61]  J. Raymond,et al.  Value associations of irrelevant stimuli modify rapid visual orienting , 2010, Psychonomic bulletin & review.

[62]  Manabu Iguchi,et al.  A bouncing droplet on an inclined hydrophobic surface , 2010, J. Vis..

[63]  Pietro Perona,et al.  Optimal reward harvesting in complex perceptual environments , 2010, Proceedings of the National Academy of Sciences.

[64]  Jon Driver,et al.  Fortune and reversals of fortune in visual search: Reward contingencies for pop-out targets affect search efficiency and target repetition effects , 2010, Attention, perception & psychophysics.

[65]  Jan Theeuwes,et al.  Please Scroll down for Article Visual Cognition Reward Has a Residual Impact on Target Selection in Visual Search, but Not on the Suppression of Distractors , 2022 .

[66]  C. Stark,et al.  Pattern separation in the hippocampus , 2011, Trends in Neurosciences.

[67]  Steven Yantis,et al.  Learned Value Magnifies Salience-Based Attentional Capture , 2011, PloS one.

[68]  Patryk A. Laurent,et al.  Value-driven attentional capture , 2011, Proceedings of the National Academy of Sciences.

[69]  Leonardo Chelazzi,et al.  Dissociable Effects of Reward on Attentional Learning: From Passive Associations to Active Monitoring , 2011, PloS one.

[70]  T. Egner,et al.  The Neural Underpinnings of How Reward Associations Can Both Guide and Misguide Attention , 2011, The Journal of Neuroscience.

[71]  Dino J. Levy,et al.  The root of all value: a neural common currency for choice , 2012, Current Opinion in Neurobiology.

[72]  Steven Yantis,et al.  Value-driven attentional and oculomotor capture during goal-directed, unconstrained viewing , 2012, Attention, Perception, & Psychophysics.

[73]  Ilya E. Monosov,et al.  What and Where Information in the Caudate Tail Guides Saccades to Visual Objects , 2012, The Journal of Neuroscience.

[74]  J. Theeuwes,et al.  Reward grabs the eye: Oculomotor capture by rewarding stimuli , 2012, Vision Research.

[75]  Howard E Egeth,et al.  The ignoring paradox: Cueing distractor features leads first to selection, then to inhibition of to-be-ignored items , 2012, Attention, perception & psychophysics.

[76]  J. Theeuwes,et al.  Top-down versus bottom-up attentional control: a failed theoretical dichotomy , 2012, Trends in Cognitive Sciences.

[77]  W. Zoest,et al.  Reward creates oculomotor salience , 2012, Current Biology.

[78]  Patryk A. Laurent,et al.  Generalization of value-based attentional priority , 2012, Visual cognition.

[79]  A. Song,et al.  The involvement of the dopaminergic midbrain and cortico-striatal-thalamic circuits in the integration of reward prospect and attentional task demands. , 2012, Cerebral cortex.

[80]  R. Krauzlis,et al.  Superior colliculus and visual spatial attention. , 2013, Annual review of neuroscience.

[81]  Christopher J. Peck,et al.  The primate amygdala combines information about space and value , 2013, Nature Neuroscience.

[82]  Perceptual gating of value-driven attentional capture , 2013 .

[83]  Steven Yantis,et al.  Reward predictions bias attentional selection , 2013, Front. Hum. Neurosci..

[84]  L. Chelazzi,et al.  Rewards teach visual selective attention , 2013, Vision Research.

[85]  Hongbo Yu,et al.  Interaction between value and perceptual salience in value-driven attentional capture. , 2013, Journal of vision.

[86]  Yuhong V Jiang,et al.  Rapid acquisition but slow extinction of an attentional bias in space. , 2012, Journal of experimental psychology. Human perception and performance.

[87]  K. Nelissen,et al.  Dopaminergic Reward Signals Selectively Decrease fMRI Activity in Primate Visual Cortex , 2013, Neuron.

[88]  W. Zoest,et al.  Reward-associated stimuli capture the eyes in spite of strategic attentional set , 2013, Vision Research.

[89]  N. Turk-Browne,et al.  Attention Is Spontaneously Biased Toward Regularities , 2013, Psychological science.

[90]  Hyoung F. Kim,et al.  Why skill matters , 2013, Trends in Cognitive Sciences.

[91]  K. Berridge,et al.  Instant Transformation of Learned Repulsion into Motivational “Wanting” , 2013, Current Biology.

[92]  Steven Yantis,et al.  Attentional bias for nondrug reward is magnified in addiction. , 2013, Experimental and clinical psychopharmacology.

[93]  B. Anderson A value-driven mechanism of attentional selection. , 2013, Journal of vision.

[94]  Jan Theeuwes,et al.  Feature-based attention: it is all bottom-up priming , 2013, Philosophical Transactions of the Royal Society B: Biological Sciences.

[95]  Shinya Yamamoto,et al.  Reward Value-Contingent Changes of Visual Responses in the Primate Caudate Tail Associated with a Visuomotor Skill , 2013, The Journal of Neuroscience.

[96]  Carol A. Seger,et al.  The visual corticostriatal loop through the tail of the caudate: circuitry and function , 2013, Front. Syst. Neurosci..

[97]  Eva Zita Patai,et al.  Reward Associations Magnify Memory-based Biases on Perception , 2013, Journal of Cognitive Neuroscience.

[98]  Senqing Qi,et al.  Neural correlates of reward-driven attentional capture in visual search , 2013, Brain Research.

[99]  S. Yantis,et al.  Persistence of value-driven attentional capture. , 2013, Journal of experimental psychology. Human perception and performance.

[100]  H. Critchley,et al.  Acute Alcohol Effects on Attentional Bias are Mediated by Subcortical Areas Associated with Arousal and Salience Attribution , 2013, Neuropsychopharmacology.

[101]  Anthony W. Sali,et al.  The role of reward prediction in the control of attention. , 2014, Journal of experimental psychology. Human perception and performance.

[102]  C. N. Boehler,et al.  Reward- and Attention-related Biasing of Sensory Selection in Visual Cortex , 2014, Journal of Cognitive Neuroscience.

[103]  Steven Yantis,et al.  Value-driven attentional priority signals in human basal ganglia and visual cortex , 2014, Brain Research.

[104]  J. Geng,et al.  Reward associations and spatial probabilities produce additive effects on attentional selection , 2014, Attention, perception & psychophysics.

[105]  Jatin G Vaidya,et al.  Value-Driven Attentional Capture in Adolescence , 2014, Psychological science.

[106]  S. Delplanque,et al.  Where is the chocolate? Rapid spatial orienting toward stimuli associated with primary rewards , 2014, Cognition.

[107]  Christopher J. Peck,et al.  The Amygdala and Basal Forebrain as a Pathway for Motivationally Guided Attention , 2014, The Journal of Neuroscience.

[108]  Raymond J. Dolan,et al.  The human amygdala encodes value and space during decision making , 2014, NeuroImage.

[109]  Steven Yantis,et al.  Attentional Bias for Non-drug Reward is Magnified in Addiction , 2014 .

[110]  D. Wentura,et al.  Attentional capture by evaluative stimuli: Gain- and loss-connoting colors boost the additional-singleton effect , 2014, Psychonomic bulletin & review.

[111]  L. Chelazzi,et al.  Altering Spatial Priority Maps via Reward-Based Learning , 2014, The Journal of Neuroscience.

[112]  The attribution of value-based attentional priority in individuals with depressive symptoms , 2014, Cognitive, affective & behavioral neuroscience.

[113]  R. Davidson,et al.  A mind you can count on: validating breath counting as a behavioral measure of mindfulness , 2014, Front. Psychol..

[114]  Christopher J. Peck,et al.  Amygdala neural activity reflects spatial attention towards stimuli promising reward or threatening punishment , 2014, eLife.

[115]  Sarah Shomstein,et al.  Reward-Based Transfer From Bottom-Up to Top-Down Search Tasks , 2013, Psychological science.

[116]  Jan Theeuwes,et al.  Reward-Priming of Location in Visual Search , 2014, PloS one.

[117]  Mengyuan Gong,et al.  Learned reward association improves visual working memory. , 2014, Journal of experimental psychology. Human perception and performance.

[118]  Steven Yantis,et al.  Learned states of preparatory attentional control. , 2015, Journal of experimental psychology. Learning, memory, and cognition.

[119]  L. Pessoa,et al.  Reward learning and negative emotion during rapid attentional competition , 2015, Front. Psychol..

[120]  B. Anderson Value-driven attentional capture is modulated by spatial context , 2015, Visual cognition.

[121]  J. Serences,et al.  Value-based attentional capture influences context-dependent decision-making. , 2015, Journal of neurophysiology.

[122]  A. Nobre,et al.  Reward boosts working memory encoding over a brief temporal window , 2015, SSRN Electronic Journal.

[123]  Jan Theeuwes,et al.  Attentional capture by signals of threat , 2014, Cognition & emotion.

[124]  Steven B. Most,et al.  Cognitive control and counterproductive oculomotor capture by reward-related stimuli , 2015 .

[125]  L. Pessoa Multiple influences of reward on perception and attention , 2015, Visual cognition.

[126]  Mary H. MacLean,et al.  Irrelevant reward and selection histories have different influences on task-relevant attentional selection , 2015, Attention, perception & psychophysics.

[127]  J. Theeuwes,et al.  Potential threat attracts attention and interferes with voluntary saccades. , 2015, Emotion.

[128]  Michel F. Failing,et al.  Nonspatial attentional capture by previously rewarded scene semantics , 2015 .

[129]  C. Hickey,et al.  Reward priming of temporal preparation , 2015 .

[130]  J. Saiki,et al.  Task-irrelevant stimulus-reward association induces value-driven attentional capture , 2015, Attention, perception & psychophysics.

[131]  Patryk A. Laurent,et al.  Valuable orientations capture attention , 2015, Visual cognition.

[132]  Brian A. Anderson,et al.  Value-driven attentional priority is context specific , 2014, Psychonomic Bulletin & Review.

[133]  Pieter R. Roelfsema,et al.  A learning rule that explains how rewards teach attention , 2015 .

[134]  Jason P. Gallivan,et al.  Separating value from selection frequency in rapid reaching biases to visual targets , 2015 .

[135]  Barry Giesbrecht,et al.  Neural evidence reveals the rapid effects of reward history on selective attention , 2015, Brain Research.

[136]  M. Peelen,et al.  Neural Mechanisms of Incentive Salience in Naturalistic Human Vision , 2015, Neuron.

[137]  Jeff Moher,et al.  Dissociable Effects of Salience on Attention and Goal-Directed Action , 2015, Current Biology.

[138]  Marius V Peelen,et al.  Reward guides attention to object categories in real-world scenes. , 2015, Journal of experimental psychology. General.

[139]  Kan Zhang,et al.  Social comparison modulates reward-driven attentional capture , 2015, Psychonomic bulletin & review.

[140]  Mircea Ariel Schoenfeld,et al.  The modulatory impact of reward and attention on global feature selection in human visual cortex , 2015 .

[141]  J. Geng,et al.  The modulation of reward priority by top-down knowledge , 2015 .

[142]  Todd A. Hare,et al.  Automatic versus Choice-Dependent Value Representations in the Human Brain , 2015, Neuron.

[143]  Andri S. Björnsson,et al.  Money talks in attention bias modification: Reward in a dot-probe task affects attentional biases , 2015 .

[144]  M. Turatto,et al.  Reward associations impact both iconic and visual working memory , 2015, Vision Research.

[145]  J. Theeuwes,et al.  Distractors that signal reward attract the eyes , 2015 .

[146]  Tom Beesley,et al.  When goals conflict with values: counterproductive attentional and oculomotor capture by reward-related stimuli. , 2015, Journal of experimental psychology. General.

[147]  Sharif I. Kronemer,et al.  Reward, attention, and HIV-related risk in HIV+ individuals , 2016, Neurobiology of Disease.

[148]  Dean F. Wong,et al.  The Role of Dopamine in Value-Based Attentional Orienting , 2016, Current Biology.

[149]  B. Anderson Value-driven attentional capture in the auditory domain , 2016, Attention, perception & psychophysics.

[150]  B. Anderson Social reward shapes attentional biases , 2016, Cognitive neuroscience.

[151]  S. Huettel,et al.  Cortical Brain Activity Reflecting Attentional Biasing Toward Reward-Predicting Cues Covaries with Economic Decision-Making Performance. , 2016, Cerebral cortex.