The Geometry of Abstraction in the Hippocampus and Prefrontal Cortex

The curse of dimensionality plagues models of reinforcement learning and decision making. The process of abstraction solves this by constructing variables describing features shared by different instances, reducing dimensionality and enabling generalization in novel situations. Here, we characterized neural representations in monkeys performing a task described by different hidden and explicit variables. Abstraction was defined operationally using the generalization performance of neural decoders across task conditions not used for training, which requires a particular geometry of neural representations. Neural ensembles in prefrontal cortex, hippocampus, and simulated neural networks simultaneously represented multiple variables in a geometry reflecting abstraction but that still allowed a linear classifier to decode a large number of other variables (high shattering dimensionality). Furthermore, this geometry changed in relation to task events and performance. These findings elucidate how the brain and artificial systems represent variables in an abstract format while preserving the advantages conferred by high shattering dimensionality.

[1]  David J. Freedman,et al.  Neural correlates of categories and concepts , 2003, Current Opinion in Neurobiology.

[2]  Stefano Fusi,et al.  Hebbian Learning in a Random Network Captures Selectivity Properties of the Prefrontal Cortex , 2017, The Journal of Neuroscience.

[3]  R. Bellman Dynamic programming. , 1957, Science.

[4]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[5]  David D. Cox,et al.  Untangling invariant object recognition , 2007, Trends in Cognitive Sciences.

[6]  L. Frank,et al.  Single Neurons in the Monkey Hippocampus and Learning of New Associations , 2003, Science.

[7]  Rudolf Stark,et al.  Imagined and Executed Actions in the Human Motor System: Testing Neural Similarity Between Execution and Imagery of Actions with a Multivariate Approach , 2016, Cerebral cortex.

[8]  Xiao-Jing Wang,et al.  Internal Representation of Task Rules by Recurrent Dynamics: The Importance of the Diversity of Neural Responses , 2010, Front. Comput. Neurosci..

[9]  D. Hassabis,et al.  Tracking the Emergence of Conceptual Knowledge during Human Decision Making , 2009, Neuron.

[11]  David J. Freedman,et al.  Dynamic population coding of category information in inferior temporal and prefrontal cortex. , 2008, Journal of neurophysiology.

[12]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[13]  E. Kandel,et al.  Cognitive Neuroscience and the Study of Memory , 1998, Neuron.

[14]  Christos Constantinidis,et al.  Emergence of Nonlinear Mixed Selectivity in Prefrontal Cortex after Training , 2020, The Journal of Neuroscience.

[15]  Roger B. Grosse,et al.  Isolating Sources of Disentanglement in Variational Autoencoders , 2018, NeurIPS.

[16]  Xinjiao Chen Confidence Interval for the Mean of a Bounded Random Variable and Its Applications in Point Estimation , 2008, 0802.3458.

[17]  Christian K. Machens,et al.  Behavioral / Systems / Cognitive Functional , But Not Anatomical , Separation of “ What ” and “ When ” in Prefrontal Cortex , 2009 .

[18]  Caswell Barry,et al.  The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation , 2019, Cell.

[19]  Matthew Botvinick,et al.  On the importance of single directions for generalization , 2018, ICLR.

[20]  Y Kamitani,et al.  Neural Decoding of Visual Imagery During Sleep , 2013, Science.

[21]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[22]  Sayan Mukherjee,et al.  Permutation Tests for Classification , 2005, COLT.

[23]  K. C. Anderson,et al.  Single neurons in prefrontal cortex encode abstract rules , 2001, Nature.

[24]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[25]  H. Eichenbaum Hippocampus Cognitive Processes and Neural Representations that Underlie Declarative Memory , 2004, Neuron.

[26]  C. Salzman,et al.  Shared neural coding for social hierarchy and reward value in primate amygdala , 2018, Nature Neuroscience.

[27]  Andriy Mnih,et al.  Disentangling by Factorising , 2018, ICML.

[28]  Matthew T. Kaufman,et al.  A category-free neural population supports evolving demands during decision-making , 2014, Nature Neuroscience.

[29]  Stefano Fusi,et al.  Are place cells just memory cells? Memory compression leads to spatial tuning and history dependence , 2019 .

[30]  C. Salzman,et al.  Abstract Context Representations in Primate Amygdala and Prefrontal Cortex , 2015, Neuron.

[31]  Nicole C. Rust,et al.  Selectivity and Tolerance (“Invariance”) Both Increase as Visual Information Propagates from Cortical Area V4 to IT , 2010, The Journal of Neuroscience.

[32]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[33]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[34]  Thomas G. Dietterich Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[37]  Doris Y. Tsao,et al.  The Code for Facial Identity in the Primate Brain , 2017, Cell.

[38]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[39]  H. Eichenbaum On the Integration of Space, Time, and Memory , 2017, Neuron.

[40]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.

[41]  Kenneth D. Harris,et al.  High-dimensional geometry of population responses in visual cortex , 2019, Nat..

[42]  M. Botvinick,et al.  Statistical learning of temporal community structure in the hippocampus , 2016, Hippocampus.

[43]  Stefano Fusi,et al.  Why neurons mix: high dimensionality for higher cognition , 2016, Current Opinion in Neurobiology.

[44]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[45]  Zeb Kurth-Nelson,et al.  What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior , 2018, Neuron.

[46]  Joel Z. Leibo,et al.  The dynamics of invariant object recognition in the human visual system. , 2014, Journal of neurophysiology.

[47]  Stefano Fusi,et al.  The Sparseness of Mixed Selectivity Neurons Controls the Generalization–Discrimination Trade-Off , 2013, The Journal of Neuroscience.

[48]  Tomaso Poggio,et al.  A fast, invariant representation for human action in the visual system. , 2018, Journal of neurophysiology.

[49]  Shih-Cheng Yen,et al.  Mixed selectivity morphs population codes in prefrontal cortex , 2017, Nature Neuroscience.

[50]  E. Miller,et al.  Differences between Neural Activity in Prefrontal Cortex and Striatum during Learning of Novel Abstract Categories , 2011, Neuron.

[51]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[52]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[53]  Carolyn Parkinson,et al.  A Common Cortical Metric for Spatial, Temporal, and Social Distance , 2014, The Journal of Neuroscience.

[54]  Earl K. Miller,et al.  Different Levels of Category Abstraction by Different Dynamics in Different Prefrontal Areas , 2018, Neuron.

[55]  G. La Camera,et al.  Stimuli Reduce the Dimensionality of Cortical Activity , 2015, bioRxiv.

[56]  Eric Shea-Brown,et al.  Predictive learning extracts latent space representations from sensory observations , 2019 .

[57]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[58]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[59]  Doina Precup,et al.  Temporal abstraction in reinforcement learning , 2000, ICML 2000.

[60]  Matthew E. Taylor,et al.  Abstraction and Generalization in Reinforcement Learning: A Summary and Framework , 2009, ALA.

[61]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[62]  Philip G. F. Browning,et al.  Dissociable Components of Rule-Guided Behavior Depend on Distinct Medial and Prefrontal Regions , 2009, Science.