Differential reinforcement encoding along the hippocampal long axis helps resolve the explore–exploit dilemma

[1]  Jiu Chen,et al.  An Activation Likelihood Estimation Meta-Analysis of Specific Functional Alterations in Dorsal Attention Network in Mild Cognitive Impairment , 2022, Frontiers in Neuroscience.

[2]  P. Dayan,et al.  The value of what’s to come: Neural mechanisms coupling prediction error and the utility of anticipation , 2020, Science Advances.

[3]  T. Therneau.  Mixed Effects Cox Models [R package coxme version 2.2-16] , 2020 .

[4]  Andrew R. Mitz,et al.  Subcortical Substrates of Explore-Exploit Decisions in Primates , 2019, Neuron.

[5]  Timothy E. J. Behrens,et al.  Human Replay Spontaneously Reorganizes Experience , 2019, Cell.

[6]  G. Schoenbaum.  Faculty Opinions recommendation of Viewpoints: how the hippocampus contributes to memory, navigation and cognition. , 2019, Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature.

[7]  Michael R. Meager,et al.  Hippocampal Contributions to Model-Based Planning and Spatial Memory , 2019, Neuron.

[8]  Yushan Huang,et al.  Involvement of hippocampal subfields and anterior-posterior subregions in encoding and retrieval of item, spatial, and associative memories: Longitudinal versus transverse axis , 2019, NeuroImage.

[9]  Christian F. Doeller,et al.  The Hippocampus Encodes Distances in Multidimensional Feature Space , 2019, Current Biology.

[10]  P. Dayan,et al.  The value of what’s to come: neural mechanisms coupling prediction error and reward anticipation , 2019, bioRxiv.

[11]  P. Dayan,et al.  The value of what’s to come: neural mechanisms coupling prediction error and reward anticipation , 2019, bioRxiv.

[12]  M. López-Canúl,et al.  The ventral hippocampus is required for behavioral flexibility but not for allocentric/egocentric learning , 2019, Brain Research Bulletin.

[13]  M. Hallquist,et al.  Selective maintenance of value information helps resolve the exploration/exploitation dilemma , 2017, Cognition.

[14]  Susanna Sallstroem,et al.  Functional Differentiation , 2009, Modern Condensed Matter Physics.

[15]  Jonathan D. Nelson,et al.  Generalization guides human exploration in vast decision spaces , 2017, Nature Human Behaviour.

[16]  Charley M. Wu,et al.  Searching for Rewards Like a Child Means Less Generalization and More Directed Exploration , 2018, bioRxiv.

[17]  Preston P. Thakral,et al.  Increased hippocampus to ventromedial prefrontal connectivity during the construction of episodic future events , 2018, Hippocampus.

[18]  E. Maguire,et al.  Comparing and Contrasting the Cognitive Effects of Hippocampal and Ventromedial Prefrontal Cortex Damage: A Review of Human Lesion Studies , 2017, Neuroscience.

[19]  Paul-Christian Bürkner,et al.  Advanced Bayesian Multilevel Modeling with the R Package brms , 2017, R J..

[20]  Tommy C. Blanchard,et al.  Pure correlates of exploration and exploitation in the human brain , 2017, Cognitive, Affective, & Behavioral Neuroscience.

[21]  Marcelo G Mattar,et al.  Prioritized memory access explains planning and hippocampal replay , 2017, Nature Neuroscience.

[22]  L. Nadel,et al.  Viewpoints: how the hippocampus contributes to memory, navigation and cognition , 2017, Nature Neuroscience.

[23]  Christos Davatzikos,et al.  Benchmarking of participant-level confound regression strategies for the control of motion artifact in studies of functional connectivity , 2017, NeuroImage.

[24]  Kimberly L. Stachenfeld,et al.  The hippocampus as a predictive map , 2017, Nature Neuroscience.

[25]  Kevin J. Miller,et al.  Dorsal hippocampus contributes to model-based planning , 2017, Nature Neuroscience.

[26]  M. Shapiro,et al.  Medial Prefrontal Cortex Reduces Memory Interference by Modifying Hippocampal Encoding , 2017, Neuron.

[27]  Robert C. Wilson,et al.  Charting the Expansion of Strategic Exploratory Behavior During Adolescence , 2017, Journal of experimental psychology. General.

[28]  Tommy C. Blanchard,et al.  Pure correlates of exploration and exploitation in the human brain , 2017, bioRxiv.

[29]  Stefano Palminteri,et al.  When are inter-individual brain-behavior correlations informative? , 2016, bioRxiv.

[30]  E. Kandel,et al.  Dopamine release from the locus coeruleus to the dorsal hippocampus promotes spatial learning and memory , 2016, Proceedings of the National Academy of Sciences.

[31]  S. Hanslmayr,et al.  Spatial Mnemonic Encoding: Theta Power Decreases and Medial Temporal Lobe BOLD Increases Co-Occur during the Usage of the Method of Loci , 2016, eNeuro.

[32]  Brad E. Pfeiffer,et al.  Reverse Replay of Hippocampal Place Cells Is Uniquely Modulated by Changing Reward , 2016, Neuron.

[33]  Hans Knutsson,et al.  Correction for Eklund et al., Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[34]  R. Morris,et al.  Locus coeruleus and dopaminergic consolidation of everyday memory , 2016, Nature.

[35]  Hans Knutsson,et al.  Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[36]  Mitchell H. Murdock,et al.  Dynamic changes in neural circuitry during adolescence are associated with persistent attenuation of fear memories , 2016, Nature Communications.

[37]  S. Hanslmayr,et al.  Spatial mnemonic encoding: Theta power decreases co-occur with medial temporal lobe BOLD increases during the usage of the Method of Loci , 2016, bioRxiv.

[38]  Freek E. Hoebeek,et al.  SLC26A11 (KBAT) in Purkinje Cells Is Critical for Inhibitory Transmission and Contributes to Locomotor Coordination , 2016, eNeuro.

[39]  Ilana B. Witten,et al.  Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target , 2016, Nature Neuroscience.

[40]  M. Lebreton,et al.  Assessing Inter-individual Variability in Brain-behavior Relationship Assessing Inter-individual Variability in Brain-behavior Relationship with Functional Neuroimaging , 2022 .

[41]  Thorsten Gerber,et al.  Applied Longitudinal Data Analysis Modeling Change And Event Occurrence , 2016 .

[42]  Onder Hazaroglu,et al.  Improving the precision of fMRI BOLD signal deconvolution with implications for connectivity analysis. , 2015, Magnetic resonance imaging.

[43]  Valentin Riedl,et al.  Cognitive emotion regulation enhances aversive prediction error activity while reducing emotional responses , 2015, NeuroImage.

[44]  S. Siegelbaum,et al.  The Corticohippocampal Circuit, Synaptic Plasticity, and Memory. , 2015, Cold Spring Harbor perspectives in biology.

[45]  T. Hare,et al.  Transcranial Stimulation over Frontopolar Cortex Elucidates the Choice Attributes and Neural Mechanisms Used to Resolve Exploration–Exploitation Trade-Offs , 2015, The Journal of Neuroscience.

[46]  Matthew F.S. Rushworth,et al.  Contrasting Roles for Orbitofrontal Cortex and Amygdala in Credit Assignment and Learning in Macaques , 2015, Neuron.

[47]  Alberto Llera,et al.  ICA-AROMA: A robust ICA-based strategy for removing motion artifacts from fMRI data , 2015, NeuroImage.

[48]  S. Eickhoff,et al.  Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis , 2015, Cognitive, affective & behavioral neuroscience.

[49]  R. Poldrack.  Is “efficiency” a useful concept in cognitive neuroscience? , 2014, Developmental Cognitive Neuroscience.

[50]  Shannon M. Pruden,et al.  A Meta-analysis of Neuroimaging Studies , 2015 .

[51]  E. Lein,et al.  Functional organization of the hippocampal longitudinal axis , 2014, Nature Reviews Neuroscience.

[52]  D. Dupret,et al.  Dopaminergic neurons promote hippocampal reactivation and spatial memory persistence , 2014, Nature Neuroscience.

[53]  Raphael T. Gerraty,et al.  Transfer of Learning Relates to Intrinsic Connectivity between Hippocampus, Ventromedial Prefrontal Cortex, and Large-Scale Networks , 2014, The Journal of Neuroscience.

[54]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[55]  Anjali Krishnan,et al.  Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations , 2014, NeuroImage.

[56]  Lionel Rigoux,et al.  VBA: A Probabilistic Treatment of Nonlinear Models for Neurobiological and Behavioural Data , 2014, PLoS Comput. Biol..

[57]  Valcir João da Cunha Farias,et al.  Analysis of Functional Magnetic Resonance Images , 2014 .

[58]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[59]  H. Eichenbaum,et al.  Interplay of Hippocampus and Prefrontal Cortex in Memory , 2013, Current Biology.

[60]  Jane R. Garrison,et al.  Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies , 2013, Neuroscience & Biobehavioral Reviews.

[61]  A. Graybiel,et al.  Prolonged Dopamine Signalling in Striatum Signals Proximity and Value of Distant Rewards , 2013, Nature.

[62]  M. Mallar Chakravarty,et al.  A novel in vivo atlas of human hippocampal subfields using high-resolution 3T magnetic resonance imaging , 2013, NeuroImage.

[63]  Keith Bush,et al.  Decoding neural events from fMRI BOLD signal: a comparison of existing approaches and development of a new algorithm. , 2013, Magnetic resonance imaging.

[64]  Howard Eichenbaum,et al.  Ventral Hippocampal Neurons Are Shaped by Experience to Represent Behaviorally Relevant Contexts , 2013, The Journal of Neuroscience.

[65]  Hallvard Røe Evensmoen,et al.  Long-axis specialization of the human hippocampus , 2013, Trends in Cognitive Sciences.

[66]  Hongkui Zeng,et al.  Differential Control of Learning and Anxiety along the Dorsoventral Axis of the Dentate Gyrus , 2013, Neuron.

[67]  Daeyeol Lee,et al.  Hippocampal Neural Correlates for Values of Experienced Events , 2012, The Journal of Neuroscience.

[68]  D. Shohamy,et al.  Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions , 2012, Science.

[69]  P. Glimcher,et al.  Adolescents’ risk-taking behavior is driven by tolerance to ambiguity , 2012, Proceedings of the National Academy of Sciences.

[70]  Elliot A. Ludvig,et al.  Evaluating the TD model of classical conditioning , 2012, Learning & behavior.

[71]  G. Buzsáki,et al.  Traveling Theta Waves along the Entire Septotemporal Axis of the Hippocampus , 2012, Neuron.

[72]  Nicole M. Long,et al.  Supplemental Figure , 2013 .

[73]  Christian F. Doeller,et al.  Movement-Related Theta Rhythm in Humans: Coordinating Self-Directed Hippocampal Learning , 2012, PLoS biology.

[74]  Dirk Ifenthaler,et al.  Stochastic Models of Learning , 2012 .

[75]  Paul M. Thompson,et al.  Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods , 2011, IEEE Transactions on Medical Imaging.

[76]  Christian F. Doeller,et al.  Anterior Hippocampus and Goal-Directed Spatial Decision Making , 2011, The Journal of Neuroscience.

[77]  Alexis Roche,et al.  A Four-Dimensional Registration Algorithm With Application to Joint Correction of Motion and Slice Timing in fMRI , 2011, IEEE Transactions on Medical Imaging.

[78]  H. Eichenbaum,et al.  Memory for the Order of Events in Specific Sequences: Contributions of the Hippocampus and Medial Prefrontal Cortex , 2011, The Journal of Neuroscience.

[79]  Jian Li,et al.  Parallel contributions of distinct human memory systems during probabilistic learning , 2011, NeuroImage.

[80]  Andre Chevrier,et al.  Error detection in the stop signal task , 2010, NeuroImage.

[81]  G. Buzsáki,et al.  Distinct Representations and Theta Dynamics in Dorsal and Ventral Hippocampus , 2010, The Journal of Neuroscience.

[82]  Mihir A. Topiwala,et al.  Synchronized Activity between the Ventral Hippocampus and the Medial Prefrontal Cortex during Anxiety , 2010, Neuron.

[83]  Hong-wei Dong,et al.  Are the Dorsal and Ventral Hippocampus Functionally Distinct Structures? , 2010, Neuron.

[84]  Bruce Fischl,et al.  Accurate and robust brain image alignment using boundary-based registration , 2009, NeuroImage.

[85]  J. Tanaka,et al.  The NimStim set of facial expressions: Judgments from untrained research participants , 2009, Psychiatry Research.

[86]  B. McNaughton,et al.  Hippocampus Leads Ventral Striatum in Replay of Place-Reward Information , 2009, PLoS biology.

[87]  C. Almli,et al.  Unbiased nonlinear average age-appropriate brain templates from birth to adulthood , 2009, NeuroImage.

[88]  Evgueniy V. Lubenov,et al.  Hippocampal theta oscillations are travelling waves , 2009, Nature.

[89]  M. Frank,et al.  The neurogenetics of exploration and exploitation: Prefrontal and striatal dopaminergic components , 2009, Nature Neuroscience.

[90]  L. Jarrard,et al.  Contributions of the hippocampus and medial prefrontal cortex to energy and body weight regulation , 2009, Hippocampus.

[91]  Arne D. Ekstrom,et al.  Correlation between BOLD fMRI and theta-band local field potentials in the human hippocampal area. , 2009, Journal of neurophysiology.

[92]  Richard S. Sutton,et al.  Stimulus Representation and the Timing of Reward-Prediction Errors in Models of the Dopamine System , 2008, Neural Computation.

[93]  Michael X. Cohen,et al.  A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism , 2008, The Journal of Neuroscience.

[94]  W. Newsome,et al.  The temporal precision of reward prediction in dopamine neurons , 2008, Nature Neuroscience.

[95]  T. Hafting,et al.  Finite Scale of Spatial Representation in the Hippocampus , 2008, Science.

[96]  Mark W. Woolrich,et al.  Robust group analysis using outlier inference , 2008, NeuroImage.

[97]  T. Hothorn,et al.  Simultaneous Inference in General Parametric Models , 2008, Biometrical journal. Biometrische Zeitschrift.

[98]  K. Jarrod Millman,et al.  Analysis of Functional Magnetic Resonance Imaging in Python , 2007, Computing in Science & Engineering.

[99]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[100]  A. David Redish,et al.  Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model , 2005, Neural Networks.

[101]  E. Rolls,et al.  Reward-Spatial View Representations and Learning in the Primate Hippocampus , 2005, The Journal of Neuroscience.

[102]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[103]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[104]  Mark W. Woolrich,et al.  Multilevel linear modelling for FMRI group analysis using Bayesian inference , 2004, NeuroImage.

[105]  J. Singer,et al.  Applied Longitudinal Data Analysis , 2003 .

[106]  Wulfram Gerstner,et al.  Reinforcement Learning in Continuous State and Action Space , 2003 .

[107]  Stephen M. Smith,et al.  Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data , 2001, NeuroImage.

[108]  B. Balleine,et al.  The Role of the Hippocampus in Instrumental Conditioning , 2000, The Journal of Neuroscience.

[109]  M. Moser,et al.  Functional differentiation in the hippocampus , 1998, Hippocampus.

[110]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[111]  B. McNaughton,et al.  Comparison of spatial firing characteristics of units in dorsal and ventral hippocampus of the rat , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[112]  Michael Recce,et al.  A model of hippocampal function , 1994, Neural Networks.

[113]  W Revelle,et al.  Very Simple Structure: An Alternative Procedure For Estimating The Optimal Number Of Interpretable Factors. , 1979, Multivariate behavioral research.

[114]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .