Exploration Disrupts Choice-Predictive Signals and Alters Dynamics in Prefrontal Cortex

[1]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[2]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[3]  H. Barbas,et al.  Organization of afferent input to subdivisions of area 8 in the rhesus monkey , 1981, The Journal of comparative neurology.

[4]  C. Bruce,et al.  Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. , 1985, Journal of neurophysiology.

[5]  C. Bruce,et al.  Primate frontal eye fields. I. Single neurons discharging before saccades. , 1985, Journal of neurophysiology.

[6]  J. Pearce,et al.  The orienting response as an index of stimulus associability in rats. , 1988, Journal of experimental psychology. Animal behavior processes.

[7]  C. Bruce,et al.  Frontal eye field efferents in the macaque monkey: II. Topography of terminal fields in midbrain and pons , 1988, The Journal of comparative neurology.

[8]  P. Goldman-Rakic,et al.  Visuospatial coding in primate prefrontal neurons revealed by oculomotor paradigms. , 1990, Journal of neurophysiology.

[9]  M. Schlag-Rey,et al.  How the frontal eye field can impose a saccade goal on superior colliculus neurons. , 1992, Journal of neurophysiology.

[10]  Jeffrey D. Schall,et al.  Neural basis of saccade target selection in frontal eye field during visual search , 1993, Nature.

[11]  C. Bruce,et al.  Topography of projections to posterior cortical areas from the macaque frontal eye fields , 1995, The Journal of comparative neurology.

[12]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[13]  M. Goldberg,et al.  Spatial processing in the monkey frontal eye field. I. Predictive visual responses. , 1997, Journal of neurophysiology.

[14]  Leslie G. Ungerleider,et al.  Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI. , 1998, Science.

[15]  M. Shadlen,et al.  Effect of Expected Reward Magnitude on the Response of Neurons in the Dorsolateral Prefrontal Cortex of the Macaque , 1999, Neuron.

[16]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[17]  J. Schall,et al.  Neural selection and control of visually guided eye movements. , 1999, Annual review of neuroscience.

[18]  T Moore,et al.  Control of eye movements and spatial attention. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[20]  O. Hikosaka,et al.  Modulation of saccadic eye movements by predicted reward outcome , 2001, Experimental Brain Research.

[21]  J. Pearce,et al.  Theories of associative learning in animals. , 2001, Annual review of psychology.

[22]  W. Bair,et al.  Correlated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior , 2001, The Journal of Neuroscience.

[23]  O. Hikosaka,et al.  Visual and Anticipatory Bias in Three Cortical Eye Fields of the Monkey during an Adaptive Decision-Making Task , 2002, The Journal of Neuroscience.

[24]  Katherine M. Armstrong,et al.  Selective gating of visual signals by microstimulation of frontal cortex , 2003, Nature.

[25]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[26]  M. Roesch,et al.  Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. , 2003, Journal of neurophysiology.

[27]  Emilio Bizzi,et al.  Discharge of frontal eye field neurons during saccadic and following eye movements in unanesthetized monkeys , 1968, Experimental Brain Research.

[28]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[29]  T. Moore,et al.  Microstimulation of the frontal eye field and its effects on covert spatial attention. , 2004, Journal of neurophysiology.

[30]  O. Hikosaka,et al.  Reward-predicting activity of dopamine and caudate neurons--a possible mechanism of motivational control of saccadic eye movement. , 2004, Journal of neurophysiology.

[31]  Takashi R Sato,et al.  Neuronal Basis of Covert Spatial Attention in the Frontal Eye Field , 2005, The Journal of Neuroscience.

[32]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[33]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[34]  P. Glimcher,et al.  JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2005, 84, 555–579 NUMBER 3(NOVEMBER) DYNAMIC RESPONSE-BY-RESPONSE MODELS OF MATCHING BEHAVIOR IN RHESUS MONKEYS , 2022 .

[35]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[36]  Xiao-Jing Wang,et al.  A Recurrent Network Mechanism of Time Integration in Perceptual Decisions , 2006, The Journal of Neuroscience.

[37]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[38]  O. Hikosaka,et al.  Comparison of Reward Modulation in the Frontal Eye Field and Caudate of the Macaque , 2006, The Journal of Neuroscience.

[39]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[40]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[41]  M. Roesch,et al.  Neuronal Activity Related to Anticipated Reward in Frontal Cortex , 2007, Annals of the New York Academy of Sciences.

[42]  P. Roelfsema,et al.  Bottom-Up Dependent Gating of Frontal Signals in Early Visual Cortex , 2008, Science.

[43]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[44]  Colin Camerer,et al.  Explicit neural signals reflecting reward uncertainty , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[45]  E. Procyk,et al.  Behavioral Shifts and Action Valuation in the Anterior Cingulate Cortex , 2008, Neuron.

[46]  John M. Pearson,et al.  Neurons in Posterior Cingulate Cortex Signal Exploratory Decisions in a Dynamic Multioption Choice Task , 2009, Current Biology.

[47]  Tirin Moore,et al.  Selection and Maintenance of Spatial Information by Frontal Eye Field Neurons , 2009, The Journal of Neuroscience.

[48]  Ian Krajbich,et al.  Visual fixations and the computation and comparison of value in simple choice , 2010, Nature Neuroscience.

[49]  B. Hayden,et al.  Neurons in Anterior Cingulate Cortex Multiplex Information about Reward and Action , 2010, The Journal of Neuroscience.

[50]  D. Durstewitz,et al.  Abrupt Transitions between Prefrontal Neural Ensemble States Accompany Behavioral Transitions during Rule Learning , 2010, Neuron.

[51]  J. Maunsell,et al.  A Neuronal Population Measure of Attention Predicts Behavioral Performance on Individual Trials , 2010, The Journal of Neuroscience.

[52]  Patryk A. Laurent,et al.  Value-driven attentional capture , 2011, Proceedings of the National Academy of Sciences.

[53]  Sander Nieuwenhuis,et al.  Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory , 2011, Journal of Cognitive Neuroscience.

[54]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[55]  Paul G. Middlebrooks,et al.  Neuronal Correlates of Metacognition in Primate Frontal Cortex , 2012, Neuron.

[56]  Jeffrey D. Schall,et al.  Supplementary Eye Field during Visual Search: Salience, Cognitive Control, and Performance Monitoring , 2012, The Journal of Neuroscience.

[57]  Mattias P. Karlsson,et al.  Network Resets in Medial Prefrontal Cortex Mark the Onset of Behavioral Uncertainty , 2012, Science.

[58]  Veit Stuphorn,et al.  Supplementary Eye Field Encodes Reward Prediction Error , 2012, The Journal of Neuroscience.

[59]  W. Zoest,et al.  Reward creates oculomotor salience , 2012, Current Biology.

[60]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[61]  Jill X. O'Reilly,et al.  Making predictions in a changing world—inference, uncertainty, and learning , 2013, Front. Neurosci..

[62]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[63]  Daeyeol Lee,et al.  Cortical Signals for Rewarded Actions and Strategic Exploration , 2013, Neuron.

[64]  Jonathan D. Cohen,et al.  Humans use directed and random exploration to solve the explore-exploit dilemma. , 2014, Journal of experimental psychology. General.

[65]  Joseph T. McGuire,et al.  Functionally Dissociable Influences on Learning Rate in a Dynamic Environment , 2014, Neuron.

[66]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[67]  Bingni W. Brunton,et al.  Cortical and Subcortical Contributions to Short-Term Memory for Orienting Movements , 2015, Neuron.

[68]  Robert C. Wilson,et al.  Reinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms , 2015, The Journal of Neuroscience.

[69]  J. Tanji,et al.  Surprise signals in the supplementary eye field: rectified prediction errors drive exploration-exploitation transitions. , 2015, Journal of Neurophysiology.

[70]  Karolina M. Lempert,et al.  Modulation of Saccade Vigor during Value-Based Decision Making , 2015, The Journal of Neuroscience.

[71]  Veit Stuphorn,et al.  Sequential selection of economic good and action in medial frontal cortex of 1 macaques during value-based decisions 2 3 Running title : Sequential good and action selection during decision-making 4 5 , 2015 .

[72]  R. B. Ebitz,et al.  Neuronal Activity in Primate Dorsal Anterior Cingulate Cortex Signals Task Conflict and Predicts Adjustments in Pupil-Linked Arousal , 2015, Neuron.

[73]  Konrad Paul Kording,et al.  Role of expected reward in frontal eye field during natural scene search. , 2016, Journal of neurophysiology.

[74]  B. Hayden,et al.  Dorsal anterior cingulate: a Rorschach test for cognitive neuroscience , 2016, Nature Neuroscience.

[75]  Tirin Moore,et al.  Selective Modulation of the Pupil Light Reflex by Microstimulation of Prefrontal Cortex , 2017, The Journal of Neuroscience.