Dark Control: A Unified Account of Default Mode Function by Control Theory and Reinforcement Learning

The default mode network (DMN) is believed to subserve the baseline mental activity in humans. Its highest energy consumption compared to other brain networks and its intimate coupling with conscious awareness are both pointing to an overarching function. Many research streams support an evolutionarily adaptive role in envisioning experience to anticipate the future. The present paper proposes a process model that tries to explain how the DMN may implement continuous evaluation and prediction of the environment to guide behavior. DMN function is recast in mathematical terms of control theory and reinforcement learning based on Markov decision processes. We argue that our formal account of DMN function naturally accommodates as special cases the previously proposed cognitive accounts on (1) predictive coding, (2) semantic associations, and (3) a “sentinel” role. Moreover, this process model for the neural optimization of complex behavior in the DMN offers parsimonious explanations for recent experimental findings in animals and humans.

[1]  R. Bálint Seelenlähmung des “Schauens”, optische Ataxie, räumliche Störung der Aufmerksamkeit. pp. 51–66 , 1909 .

[2]  R. Bálint Seelenlähmung des “Schauens”, optische Ataxie, räumliche Störung der Aufmerksamkeit. pp. 67–81 , 1909 .

[3]  T. Brown On the nature of the fundamental activity of the nervous centres; together with an analysis of the conditioning of rhythmic activity in progression, and a theory of the evolution of function in the nervous system , 1914, The Journal of physiology.

[4]  J. Antrobus,et al.  Studies in the stream of consciousness: Experimental enhancement and suppression of spontaneous cognitive processes , 1966 .

[5]  L M Giambra,et al.  Daydreaming as a Function of Cueing and Task Difficulty , 1973, Perceptual and motor skills.

[6]  Kenneth S. Pope,et al.  Regulation of the Stream of Consciousness: Toward a Theory of Ongoing Thought , 1978 .

[7]  Hanna Damasio,et al.  Locus of Lesion in Impaired Pantomime Recognition , 1987, Cortex.

[8]  P S Goldman-Rakic,et al.  Development of cortical circuitry and cognitive function. , 1987, Child development.

[9]  Richard W. Byrne,et al.  The Machiavellian intelligence hypotheses: Editorial. , 1988 .

[10]  A. Baddeley,et al.  Stimulus-independent thought depends on central executive resources , 1995, Memory & cognition.

[11]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[12]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[13]  P A Salin,et al.  Corticocortical connections in the visual system: structure and function. , 1995, Physiological reviews.

[14]  E. Lynd-Balta,et al.  The orbital and medial prefrontal circuit through the primate basal ganglia , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  A. R. Cools,et al.  The prefrontal landscape: Implications of functional architecture for understanding human mentation and the central executive - discussion , 1996 .

[16]  A. Damasio The somatic marker hypothesis and the possible functions of the prefrontal cortex. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[17]  P. Goldman-Rakic The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[18]  Robin I. M. Dunbar,et al.  Human conversational behavior , 1997, Human nature.

[19]  M. Corbetta,et al.  Common Blood Flow Changes across Visual Tasks: II. Decreases in Cerebral Cortex , 1997, Journal of Cognitive Neuroscience.

[20]  M. Mesulam,et al.  From sensation to cognition. , 1998, Brain : a journal of neurology.

[21]  E. Koechlin,et al.  The role of the anterior prefrontal cortex in human cognition , 1999, Nature.

[22]  J. A. Frost,et al.  Conceptual Processing during the Conscious Resting State: A Functional MRI Study , 1999, Journal of Cognitive Neuroscience.

[23]  D. Margoliash,et al.  Song replay during sleep and computational rules for sensorimotor vocal learning. , 2000, Science.

[24]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[25]  Richard S. J. Frackowiak,et al.  Navigation-related structural change in the hippocampi of taxi drivers. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[26]  E. Koechlin,et al.  Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[27]  T. Shallice,et al.  The cognitive and neuroanatomical correlates of multitasking , 2000, Neuropsychologia.

[28]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[29]  R. Dolan,et al.  Psychology: Reward value of attractiveness and gaze , 2001, Nature.

[30]  J. Stevenson The cultural origins of human cognition , 2001 .

[31]  G. Shulman,et al.  Medial prefrontal cortex and self-referential mental activity: Relation to a default mode of brain function , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[32]  M. Raichle,et al.  Searching for a baseline: Functional imaging and the resting human brain , 2001, Nature Reviews Neuroscience.

[33]  E. Murray,et al.  The amygdala and reward , 2002, Nature Reviews Neuroscience.

[34]  O. Blanke,et al.  Neuropsychology: Stimulating illusory own-body perceptions , 2002, Nature.

[35]  T. Braver,et al.  The Role of Frontopolar Cortex in Subgoal Processing during Working Memory , 2002, NeuroImage.

[36]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[37]  A. Grinvald,et al.  Spontaneously emerging cortical representations of visual attributes , 2003, Nature.

[38]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[39]  L. Green,et al.  A discounting framework for choice with delayed and probabilistic rewards. , 2004, Psychological bulletin.

[40]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.

[41]  Geoffrey E. Hinton,et al.  Reinforcement Learning with Factored States and Actions , 2004, J. Mach. Learn. Res..

[42]  E. Rolls,et al.  The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology , 2004, Progress in Neurobiology.

[43]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[44]  G. Buzsáki Large-scale recording of neuronal ensembles , 2004, Nature Neuroscience.

[45]  M. Weliky,et al.  Small modulation of ongoing cortical dynamics by sensory input during natural vision , 2004, Nature.

[46]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[47]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[48]  Timothy Edward John Behrens,et al.  Quantitative Investigation of Connections of the Prefrontal Cortex in the Human and Macaque using Probabilistic Diffusion Tractography , 2005, The Journal of Neuroscience.

[49]  M. Platt,et al.  Risk-sensitive neurons in macaque posterior cingulate cortex , 2005, Nature Neuroscience.

[50]  Marcus E Raichle,et al.  Intrinsic brain activity sets the stage for expression of motivated behavior , 2005, The Journal of comparative neurology.

[51]  Maurizio Corbetta,et al.  The human brain is intrinsically organized into dynamic, anticorrelated functional networks. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[52]  Michel Kerszberg,et al.  Acquisition and performance of delayed-response tasks: a neural network model. , 2005, Cerebral cortex.

[53]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[54]  F. Nielsen,et al.  Right temporoparietal cortex activation during visuo-proprioceptive conflict. , 2004, Cerebral Cortex.

[55]  J. Hermsdörfer,et al.  It takes the whole brain to make a cup of coffee: the neuropsychology of naturalistic actions involving technical devices , 2005, Neuropsychologia.

[56]  A. Cavanna,et al.  The precuneus: a review of its functional anatomy and behavioural correlates. , 2006, Brain : a journal of neurology.

[57]  Jonathan D. Cohen,et al.  Imaging valuation models in human choice. , 2006, Annual review of neuroscience.

[58]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[59]  M A H Dempster,et al.  An automated FX trading system using adaptive reinforcement learning , 2006, Expert Syst. Appl..

[60]  Benjamin J. Shannon,et al.  Coherent spontaneous activity identifies a hippocampal-parietal memory network. , 2006, Journal of neurophysiology.

[61]  R. N. Spreng,et al.  The temporal distribution of past and future autobiographical events across the lifespan , 2006, Memory & cognition.

[62]  Kristina M. Visscher,et al.  The neural bases of momentary lapses in attention , 2006, Nature Neuroscience.

[63]  B. McNaughton,et al.  EEG sharp waves and sparse ensemble unit activity in the macaque hippocampus. , 2007, Journal of neurophysiology.

[64]  T. McNamara,et al.  Different mental representations for place recognition and goal localization , 2007, Psychonomic bulletin & review.

[65]  Gereon R. Fink,et al.  Mechanisms of hemispheric specialization: Insights from analyses of connectivity , 2007, Neuropsychologia.

[66]  M. Bar,et al.  The units of thought , 2007, Hippocampus.

[67]  D. Hassabis,et al.  Patients with hippocampal amnesia cannot imagine new experiences , 2007, Proceedings of the National Academy of Sciences.

[68]  G. Buzsáki,et al.  Forward and reverse hippocampal place-cell sequences during ripples , 2007, Nature Neuroscience.

[69]  M. Corballis,et al.  The evolution of foresight: What is mental time travel, and is it unique to humans? , 2007, The Behavioral and brain sciences.

[70]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[71]  M. Bar The proactive brain: using analogies and associations to generate predictions , 2007, Trends in Cognitive Sciences.

[72]  R. Buckner,et al.  Self-projection and the brain , 2007, Trends in Cognitive Sciences.

[73]  D. Schacter,et al.  Remembering the past to imagine the future: the prospective brain , 2007, Nature Reviews Neuroscience.

[74]  Rafael Malach,et al.  Extrinsic and intrinsic systems in the posterior cortex of the human brain revealed during natural sensory stimulation. , 2007, Cerebral cortex.

[75]  Scott T. Grafton,et al.  Wandering Minds: The Default Network and Stimulus-Independent Thought , 2007, Science.

[76]  E. Stein,et al.  Cingulate activation increases dynamically with response speed under stimulus unpredictability. , 2007, Cerebral cortex.

[77]  Karl J. Friston Hierarchical Models in the Brain , 2008, PLoS Comput. Biol..

[78]  N. Burgess Spatial Cognition and the Brain , 2008, Annals of the New York Academy of Sciences.

[79]  Mark W Woolrich,et al.  Associative learning of social value , 2008, Nature.

[80]  G. Fink,et al.  Minds at rest? Social cognition as the default mode of cognizing and its putative relationship to the “default system” of the brain , 2008, Consciousness and Cognition.

[81]  Benjamin Y. Hayden,et al.  Posterior Cingulate Cortex Mediates Outcome-Contingent Allocation of Behavior , 2008, Neuron.

[82]  Russell A. Epstein Parahippocampal and retrosplenial contributions to human spatial navigation , 2008, Trends in Cognitive Sciences.

[83]  P. Dayan,et al.  Decision theory, reinforcement learning, and the brain , 2008, Cognitive, affective & behavioral neuroscience.

[84]  A. Pérez-Villalba Rhythms of the Brain, G. Buzsáki. Oxford University Press, Madison Avenue, New York (2006), Price: GB £42.00, p. 448, ISBN: 0-19-530106-4 , 2008 .

[85]  Scott T. Grafton,et al.  Action outcomes are represented in human inferior frontoparietal cortex. , 2008, Cerebral cortex.

[86]  I. Fried,et al.  Internally Generated Reactivation of Single Neurons in Human Hippocampus During Free Recall , 2008, Science.

[87]  Xiao-Jing Wang Decision Making in Recurrent Neuronal Circuits , 2008, Neuron.

[88]  P. Boyer Evolutionary economics of mental time travel? , 2008, Trends in Cognitive Sciences.

[89]  M. Corbetta,et al.  The Reorienting System of the Human Brain: From Environment to Theory of Mind , 2008, Neuron.

[90]  O. Sporns,et al.  Mapping the Structural Core of Human Cerebral Cortex , 2008, PLoS biology.

[91]  Alana T. Wong,et al.  Age-Related Changes in the Episodic Simulation of Future Events , 2008, Psychological science.

[92]  D. Paré,et al.  Contrasting Activity Profile of Two Distributed Cortical Networks as a Function of Attentional Demands , 2009, The Journal of Neuroscience.

[93]  S. Floresco,et al.  Fundamental Contribution by the Basolateral Amygdala to Different Forms of Decision Making , 2009, The Journal of Neuroscience.

[94]  A. Braun,et al.  Decoupling of the brain's default mode network during deep sleep , 2009, Proceedings of the National Academy of Sciences.

[95]  V. Michel,et al.  An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging , 2009, Neuron.

[96]  Moshe Bar,et al.  The proactive brain: memory for predictions , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[97]  Karl J. Friston,et al.  Reinforcement Learning or Active Inference? , 2009, PloS one.

[98]  R. Nathan Spreng,et al.  The Common Neural Basis of Autobiographical Memory, Prospection, Navigation, Theory of Mind, and the Default Mode: A Quantitative Meta-analysis , 2009, Journal of Cognitive Neuroscience.

[99]  Stephen M Smith,et al.  Correspondence of the brain's functional architecture during activation and rest , 2009, Proceedings of the National Academy of Sciences.

[100]  Simon B Eickhoff,et al.  Investigating the Functional Heterogeneity of the Default Mode Network Using Coordinate-Based Meta-Analytic Modeling , 2009, The Journal of Neuroscience.

[101]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[102]  John M. Pearson,et al.  Neurons in Posterior Cingulate Cortex Signal Exploratory Decisions in a Dynamic Multioption Choice Task , 2009, Current Biology.

[103]  Demis Hassabis,et al.  The construction system of the brain , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[104]  E. Maguire,et al.  What does the retrosplenial cortex do? , 2009, Nature Reviews Neuroscience.

[105]  B. Hayden,et al.  Electrophysiological correlates of default-mode processing in macaque posterior cingulate cortex , 2009, Proceedings of the National Academy of Sciences.

[106]  Simon B. Eickhoff,et al.  Effects of timing and movement uncertainty implicate the temporo-parietal junction in the prediction of forthcoming motor actions , 2009, NeuroImage.

[107]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[108]  M. Corbetta,et al.  Temporal dynamics of spontaneous MEG activity in brain networks , 2010, Proceedings of the National Academy of Sciences.

[109]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[110]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[111]  Karl J. Friston,et al.  Neural Mechanisms of Belief Inference during Cooperative Games , 2010, The Journal of Neuroscience.

[112]  M. Raichle The Brain's Dark Energy , 2006, Science.

[113]  J. O’Neill,et al.  Play it again: reactivation of waking experience and memory , 2010, Trends in Neurosciences.

[114]  Markku Penttonen,et al.  Hippocampal Ripple-Contingent Training Accelerates Trace Eyeblink Conditioning and Retards Extinction in Rabbits , 2010, The Journal of Neuroscience.

[115]  F. Turkheimer,et al.  Emergence of resting state networks in the preterm human brain , 2010, Proceedings of the National Academy of Sciences.

[116]  R. Saxe,et al.  Disruption of the right temporoparietal junction with transcranial magnetic stimulation reduces the role of beliefs in moral judgments , 2010, Proceedings of the National Academy of Sciences.

[117]  Karl J. Friston,et al.  The default-mode, ego-functions and free-energy: a neurobiological account of Freudian ideas , 2010, Brain : a journal of neurology.

[118]  John Suckling,et al.  Shared Neural Circuits for Mentalizing about the Self and Others , 2010, Journal of Cognitive Neuroscience.

[119]  R. Buckner,et al.  Functional-Anatomic Fractionation of the Brain's Default Network , 2010, Neuron.

[120]  Christian F. Doeller,et al.  Establishing the Boundaries: The Hippocampal Contribution to Imagining Scenes , 2010, The Journal of Neuroscience.

[121]  Fenna M. Krienen,et al.  Clan Mentality: Evidence That the Medial Prefrontal Cortex Responds to Close Others , 2010, The Journal of Neuroscience.

[122]  Henrik Hult,et al.  Algorithmic trading with Markov chains , 2010 .

[123]  Darren Price,et al.  Investigating the electrophysiological basis of resting state networks using magnetoencephalography , 2011, Proceedings of the National Academy of Sciences.

[124]  Rutvik H. Desai,et al.  The neurobiology of semantic memory , 2011, Trends in Cognitive Sciences.

[125]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[126]  C. Price,et al.  The neural correlates of inner speech defined by voxel-based lesion–symptom mapping , 2011, Brain : a journal of neurology.

[127]  G. Pezzulo Grounding Procedural and Declarative Knowledge in Sensorimotor Anticipation , 2011 .

[128]  Sen Cheng,et al.  Reactivation, Replay, and Preplay: How It Might All Fit Together , 2011, Neural plasticity.

[129]  G. Orban,et al.  Default Mode of Brain Function in Monkeys , 2011, The Journal of Neuroscience.

[130]  Behavior based learning in identifying High Frequency Trading strategies , 2012, 2012 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr).

[131]  P. Fox,et al.  Introspective Minds: Using ALE Meta-Analyses to Study Commonalities in the Neural Correlates of Emotional Processing, Social & Unconstrained Cognition , 2012, PloS one.

[132]  Jacqueline Nadel,et al.  Anatomical Connectivity Influences both Intra- and Inter-Brain Synchronizations , 2012, PloS one.

[133]  K. Vogeley,et al.  Parsing the neural correlates of moral cognition: ALE meta-analysis on morality, theory of mind, and empathy , 2012, Brain Structure and Function.

[134]  Jason P. Mitchell,et al.  Social-Cognitive Deficits in Normal Aging , 2012, The Journal of Neuroscience.

[135]  J. Gläscher,et al.  Lesion mapping of cognitive control and value-based decision making in the prefrontal cortex , 2012, Proceedings of the National Academy of Sciences.

[136]  M. Raichle,et al.  Rat brains also have a default mode network , 2012, Proceedings of the National Academy of Sciences.

[137]  Karl J. Friston,et al.  Free-Energy Minimization and the Dark-Room Problem , 2012, Front. Psychology.

[138]  Peter A. Beling,et al.  An agent based model of the E-Mini S&P 500 applied to flash crash analysis , 2011, 2012 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr).

[139]  J. O'Doherty,et al.  Neural Correlates of the Divergence of Instrumental Probability Distributions , 2013, The Journal of Neuroscience.

[140]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[141]  P. Fox,et al.  Segregation of the human medial prefrontal cortex in social cognition , 2013, Front. Hum. Neurosci..

[142]  Angela R. Laird,et al.  Characterization of the temporo-parietal junction by combining data-driven parcellation, complementary connectivity analyses, and functional decoding , 2013, NeuroImage.

[143]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[144]  K. Zilles,et al.  An investigation of the structural, connectional, and functional subspecialization in the human amygdala , 2012, Human brain mapping.

[145]  Benjamin J. Shannon,et al.  Morning-evening variation in human brain metabolism and memory circuits. , 2013, Journal of neurophysiology.

[146]  Krishnendu Chatterjee,et al.  Trading Performance for Stability in Markov Decision Processes , 2013, 2013 28th Annual ACM/IEEE Symposium on Logic in Computer Science.

[147]  Karl J. Friston,et al.  Computational psychiatry: the brain as a phantastic organ. , 2014, The lancet. Psychiatry.

[148]  P. Dayan,et al.  Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation , 2014, Cognitive, affective & behavioral neuroscience.

[149]  J. O’Keefe,et al.  Space in the brain: how the hippocampal formation supports spatial cognition , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[150]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[151]  Peter A. Beling,et al.  Algorithmic trading behavior identification using reward learning method , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[152]  P. Dayan,et al.  The algorithmic anatomy of model-based evaluation , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[153]  Joseph A. Maldjian,et al.  Graph theoretical analysis of resting-state MEG data: Identifying interhemispheric connectivity and the default mode , 2014, NeuroImage.

[154]  Tomonori Takeuchi,et al.  The synaptic plasticity and memory hypothesis: encoding, storage and persistence , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[155]  D. Sharp,et al.  The role of the posterior cingulate cortex in cognition and disease. , 2014, Brain : a journal of neurology.

[156]  Stephen M Smith,et al.  Fast transient networks in spontaneous human brain activity , 2014, eLife.

[157]  Angela R. Laird,et al.  Subspecialization in the human posterior medial cortex , 2015, NeuroImage.

[158]  Shie Mannor,et al.  Bayesian Reinforcement Learning: A Survey , 2015, Found. Trends Mach. Learn..

[159]  Sang Wan Lee,et al.  The structure of reinforcement-learning mechanisms in the human brain , 2015, Current Opinion in Behavioral Sciences.

[160]  Steve Y. Yang,et al.  Gaussian process-based algorithmic trading strategy identification , 2012 .

[161]  Joerg F. Hipp,et al.  BOLD fMRI Correlation Reflects Frequency-Specific Neuronal Correlation , 2015, Current Biology.

[162]  Shakir Mohamed,et al.  Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning , 2015, NIPS.

[163]  Samuel J. Gershman,et al.  Computational rationality: A converging paradigm for intelligence in brains, minds, and machines , 2015, Science.

[164]  Simon B. Eickhoff,et al.  The Resting-State Physiology of the Human Cerebral Cortex , 2015 .

[165]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[166]  H T Siegelmann,et al.  The global landscape of cognition: hierarchical aggregation as an organizational principle of human cortical networks and functions , 2015, Scientific Reports.

[167]  Karim Benchenane,et al.  Explicit memory creation during sleep demonstrates a causal role of place cells in navigation , 2015, Nature Neuroscience.

[168]  Angela R. Laird,et al.  Left inferior parietal lobe engagement in social cognition and language , 2016, Neuroscience & Biobehavioral Reviews.

[169]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[170]  Daniel L. Schacter,et al.  Mind-Wandering With and Without Intention , 2016, Trends in Cognitive Sciences.

[171]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[172]  Timothy E. J. Behrens,et al.  Organizing conceptual knowledge in humans with a gridlike code , 2016, Science.

[173]  Janice Chen,et al.  Dynamic reconfiguration of the default mode network during narrative comprehension , 2016, Nature Communications.

[174]  Zachary C. Irving,et al.  Mind-wandering as spontaneous thought: a dynamic framework , 2016, Nature Reviews Neuroscience.

[175]  E. Maguire,et al.  Anterior hippocampus: the anatomy of perception, imagination and episodic memory , 2016, Nature Reviews Neuroscience.

[176]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[177]  Danilo Bzdok,et al.  Formal Models of the Network Co-occurrence Underlying Mental Operations , 2016, PLoS Comput. Biol..

[178]  Lawrence Carin,et al.  Linear Feature Encoding for Reinforcement Learning , 2016, NIPS.

[179]  Elizabeth Jefferies,et al.  Situating the default-mode network along a principal gradient of macroscale cortical organization , 2016, Proceedings of the National Academy of Sciences.

[180]  Tobias Navarro Schröder,et al.  An event map of memory space in the hippocampus , 2016, eLife.

[181]  Kenji Doya,et al.  From free energy to expected energy: Improving energy-based value function approximation in reinforcement learning , 2016, Neural Networks.

[182]  S. Gershman,et al.  Dopamine reward prediction errors reflect hidden state inference across time , 2017, Nature Neuroscience.

[183]  Owen Y. Chao,et al.  Interaction between the medial prefrontal cortex and hippocampal CA1 area is essential for episodic-like memory in rats , 2017, Neurobiology of Learning and Memory.

[184]  Demis Hassabis,et al.  Neural Episodic Control , 2017, ICML.

[185]  Karl J. Friston,et al.  The Neural Representation of Prospective Choice during Spatial Planning and Decisions , 2017, PLoS biology.

[186]  Geraint Rees,et al.  The default network and the combination of cognitive processes that mediate self-generated thought , 2017, Nature Human Behaviour.

[187]  David K. Menon,et al.  Default mode contributions to automated information processing , 2017, Proceedings of the National Academy of Sciences.

[188]  Dmitriy Aronov,et al.  Mapping of a non-spatial dimension by the hippocampal/entorhinal circuit , 2017, Nature.

[189]  Beatrix Emo,et al.  Hippocampal and prefrontal processing of network topology to simulate the future , 2017, Nature Communications.

[190]  Tom Schaul,et al.  Rainbow: Combining Improvements in Deep Reinforcement Learning , 2017, AAAI.

[191]  Huyen Pham,et al.  Algorithmic trading in a microstructural limit order book model , 2017, Quantitative Finance.