Subcortical Substrates of Explore-Exploit Decisions in Primates

[1]  H. Klüver,et al.  PRELIMINARY ANALYSIS OF FUNCTIONS OF THE TEMPORAL LOBES IN MONKEYS , 1939 .

[2]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[3]  A. Slater,et al.  Pattern preferences at birth and their interaction with habituation-induced novelty preferences. , 1985, Journal of experimental child psychology.

[4]  Q. Vuong.  Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses , 1989 .

[5]  T. Poggio.  A theory of how the brain might work. , 1990, Cold Spring Harbor symposia on quantitative biology.

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  T. Robbins,et al.  Effects of lesions to amygdala, ventral subiculum, medial prefrontal cortex, and nucleus accumbens on the reaction to novelty: implication for limbic-striatal interactions. , 1996, Behavioral neuroscience.

[8]  B. Richmond,et al.  Neuronal Signals in the Monkey Ventral Striatum Related to Progress through a Predictable Series of Trials , 1998, The Journal of Neuroscience.

[9]  Leslie Pack Kaelbling,et al.  Algorithms for multi-armed bandit problems , 2014, ArXiv.

[10]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[11]  T. Sejnowski,et al.  Simulating a lesion in a basis function model of spatial representations: comparison with hemineglect. , 2001, Psychological review.

[12]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[13]  J. Algina,et al.  Generalized eta and omega squared statistics: measures of effect size for some common research designs. , 2003, Psychological methods.

[14]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[15]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[16]  Colin Camerer,et al.  Neural Systems Responding to Degrees of Uncertainty in Human Decision-Making , 2005, Science.

[17]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  A. Mitz.  A liquid-delivery device that provides precise reward control for neurophysiological and behavioral experiments , 2005, Journal of Neuroscience Methods.

[20]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[21]  Joseph J. Paton,et al.  The primate amygdala represents the positive and negative value of visual stimuli during learning , 2006, Nature.

[22]  D. Amaral,et al.  Amygdalectomy and responsiveness to novelty in rhesus monkeys (Macaca mulatta): generality and individual consistency of effects. , 2006, Emotion.

[23]  D. Paré,et al.  Identification of basolateral amygdala projection cells and interneurons using extracellular recordings. , 2006, Journal of neurophysiology.

[24]  A. Lüthi,et al.  Processing of Temporal Unpredictability in Human and Animal Amygdala , 2007, The Journal of Neuroscience.

[25]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[26]  B. Richmond,et al.  A Comparison of Reward‐Contingent Neuronal Activity in Monkey Orbitofrontal Cortex and Ventral Striatum , 2007, Annals of the New York Academy of Sciences.

[27]  N. Logothetis,et al.  A combined MRI and histology atlas of the rhesus monkey brain in stereotaxic coordinates , 2007 .

[28]  N. Daw,et al.  Striatal Activity Underlies Novelty-Based Choice in Humans , 2008, Neuron.

[29]  Emad N. Eskandar,et al.  Achieving behavioral control with millisecond resolution in a high-level programming environment , 2008, Journal of Neuroscience Methods.

[30]  Joseph J. Paton,et al.  Moment-to-Moment Tracking of State Value in the Amygdala , 2008, The Journal of Neuroscience.

[31]  M. Bradley.  Natural selective attention: orienting and emotion. , 2009, Psychophysiology.

[32]  John M. Pearson,et al.  Neurons in Posterior Cingulate Cortex Signal Exploratory Decisions in a Dynamic Multioption Choice Task , 2009, Current Biology.

[33]  M. Lee,et al.  A Bayesian analysis of human decision-making on bandit problems , 2009 .

[34]  Ethan S. Bromberg-Martin,et al.  Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards , 2009, Neuron.

[35]  M. Frank,et al.  Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. , 2009, Nature neuroscience.

[36]  Joshua L. Jones,et al.  Basolateral Amygdala Modulates Terminal Dopamine Release in the Nucleus Accumbens and Conditioned Responding , 2010, Biological Psychiatry.

[37]  June-Seek Choi,et al.  Amygdala regulates risk of predation in rats foraging in a dynamic fear environment , 2010, Proceedings of the National Academy of Sciences.

[38]  Sara E. Morrison,et al.  Re-valuing the amygdala , 2010, Current Opinion in Neurobiology.

[39]  S. Haber,et al.  The Reward Circuit: Linking Primate Anatomy and Human Imaging , 2010, Neuropsychopharmacology.

[40]  W. Schultz,et al.  Responses of Amygdala Neurons to Positive Reward-Predicting Stimuli Depend on Background Reward (Contingency) Rather Than Stimulus-Reward Pairing (Contiguity) , 2009, Journal of neurophysiology.

[41]  Greg O. Horne,et al.  Controlling low-level image properties: The SHINE toolbox , 2010, Behavior research methods.

[42]  B. Averbeck,et al.  Novelty seeking behaviour in Parkinson's disease , 2011, Neuropsychologia.

[43]  Alice M Stamatakis,et al.  Excitatory transmission from the amygdala to nucleus accumbens facilitates reward seeking. , 2011, Nature.

[44]  Daeyeol Lee,et al.  Heterogeneous Coding of Temporally Discounted Values in the Dorsal and Ventral Striatum during Intertemporal Choice , 2011, Neuron.

[45]  W. Schultz,et al.  Sensitivity to Temporal Reward Structure in Amygdala Neurons , 2012, Current Biology.

[46]  B. Averbeck,et al.  Uncertainty about mapping future actions into rewards may underlie performance on multiple measures of impulsivity in behavioral addiction: evidence from Parkinson's disease. , 2013, Behavioral neuroscience.

[47]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[48]  Robert C. Wilson,et al.  Orbitofrontal Cortex as a Cognitive Map of Task Space , 2014, Neuron.

[49]  Jonathan D. Cohen,et al.  Humans use directed and random exploration to solve the explore-exploit dilemma. , 2014, Journal of experimental psychology. General.

[50]  C. H. Donahue,et al.  Neural correlates of strategic reasoning during competitive games , 2014, Science.

[51]  Vincent D Costa,et al.  Dopamine modulates novelty seeking behavior during decision making. , 2014, Behavioral neuroscience.

[52]  Brianna J. Sleezer,et al.  Signatures of Value Comparison in Ventral Striatum Neurons , 2015, PLoS biology.

[53]  C. Salzman,et al.  Abstract Context Representations in Primate Amygdala and Prefrontal Cortex , 2015, Neuron.

[54]  Vincent D Costa,et al.  Imaging distributed and massed repetitions of natural scenes: Spontaneous retrieval and maintenance , 2014, Human brain mapping.

[55]  Bruno B. Averbeck,et al.  Theory of Choice in Bandit, Information Sampling and Foraging Tasks , 2015, PLoS Comput. Biol..

[56]  Tommy C. Blanchard,et al.  Orbitofrontal Cortex Uses Distinct Codes for Different Choice Attributes in Decisions Motivated by Curiosity , 2015, Neuron.

[57]  W. Schultz.  Neuronal Reward and Decision Signals: From Theories to Data. , 2015, Physiological reviews.

[58]  Ian R. Wickersham,et al.  A Circuit Mechanism for Differentiating Positive and Negative Associations , 2015, Nature.

[59]  Edmund C Schwartz,et al.  Neural Representations of Unconditioned Stimuli in Basolateral Amygdala Mediate Innate and Learned Responses , 2015, Cell.

[60]  Drew B. Headley,et al.  Amygdala Signaling during Foraging in a Hazardous Environment , 2015, The Journal of Neuroscience.

[61]  T. Hare,et al.  Transcranial Stimulation over Frontopolar Cortex Elucidates the Choice Attributes and Neural Mechanisms Used to Resolve Exploration–Exploitation Trade-Offs , 2015, The Journal of Neuroscience.

[62]  B. Hayden,et al.  The Psychology and Neuroscience of Curiosity , 2015, Neuron.

[63]  Vincent D Costa,et al.  The Role of Frontal Cortical and Medial-Temporal Lobe Brain Areas in Learning a Bayesian Prior Belief on Reversals , 2015, The Journal of Neuroscience.

[64]  Vincent D Costa,et al.  Reversal Learning and Dopamine: A Bayesian Perspective , 2015, The Journal of Neuroscience.

[65]  S. Floresco.  The nucleus accumbens: an interface between cognition, emotion, and action. , 2015, Annual review of psychology.

[66]  Ilana B. Witten,et al.  Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target , 2016, Nature Neuroscience.

[67]  William R. Stauffer,et al.  Dopamine neurons learn relative chosen value from probabilistic rewards , 2016, eLife.

[68]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[69]  Jeremiah Y. Cohen,et al.  Distributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons , 2016, Neuron.

[70]  W. Schultz,et al.  Primate amygdala neurons evaluate the progress of self-defined economic choice sequences , 2016, eLife.

[71]  Vincent D Costa,et al.  Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning , 2016, Neuron.

[72]  M. Frank,et al.  Biases in the Explore–Exploit Tradeoff in Addictions: The Role of Avoidance of Uncertainty , 2015, Neuropsychopharmacology.

[73]  John M. Pearson,et al.  A Primer on Foraging and the Explore/Exploit Trade-Off for Psychiatry Research , 2017, Neuropsychopharmacology.

[74]  P. Apicella.  The role of the intrinsic cholinergic system of the striatum: What have we learned from TAN recordings in behaving animals? , 2017, Neuroscience.

[75]  Robert C. Wilson,et al.  Charting the Expansion of Strategic Exploratory Behavior During Adolescence , 2017, Journal of experimental psychology. General.

[76]  Vincent D Costa,et al.  Motivational neural circuits underlying reinforcement learning , 2017, Nature Neuroscience.

[77]  Jonathan D. Cohen,et al.  The effect of atomoxetine on random and directed exploration in humans , 2017, PloS one.

[78]  Kathryn M. Rothenhoefer,et al.  Effects of Ventral Striatum Lesions on Stimulus-Based versus Action-Based Reinforcement Learning , 2017, The Journal of Neuroscience.

[79]  Bruno B. Averbeck,et al.  Amygdala and ventral striatum population codes implement multiple learning rates for reinforcement learning , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[80]  Robert C. Wilson,et al.  A causal role for right frontopolar cortex in directed, but not random, exploration , 2016, bioRxiv.

[81]  H. Critchley,et al.  A neurocomputational account of reward and novelty processing and effects of psychostimulants in attention deficit hyperactivity disorder , 2018, Brain : a journal of neurology.

[82]  Vincent D Costa,et al.  Ventral striatum’s role in learning from gains and losses , 2018, Proceedings of the National Academy of Sciences.

[83]  Tommy C. Blanchard,et al.  Pure correlates of exploration and exploitation in the human brain , 2017, Cognitive, Affective, & Behavioral Neuroscience.

[84]  T. Moore,et al.  Exploration Disrupts Choice-Predictive Signals and Alters Dynamics in Prefrontal Cortex , 2017, Neuron.

[85]  Drew B. Headley,et al.  Multi-dimensional Coding by Basolateral Amygdala Neurons , 2018, Neuron.

[86]  Bruno B. Averbeck,et al.  A comparison of auditory oddball responses in dorsolateral prefrontal cortex, basolateral amygdala and auditory cortex of macaque , 2018, bioRxiv.