The atoms of neural computation

Does the brain depend on a set of elementary, reusable computations? The human cerebral cortex is central to a wide array of cognitive functions, from vision to language, reasoning, decision-making, and motor control. Yet, nearly a century after the neuroanatomical organization of the cortex was first defined, its basic logic remains unknown. One hypothesis is that cortical neurons form a single, massively repeated “canonical” circuit, characterized as a kind of a “nonlinear spatiotemporal filter with adaptive properties” (1). In this classic view, it was “assumed that these…properties are identical for all neocortical areas.” Nearly four decades later, there is still no consensus about whether such a canonical circuit exists, either in terms of its anatomical basis or its function. Likewise, there is little evidence that such uniform architectures can capture the diversity of cortical function in simple mammals, let alone characteristically human processes such as language and abstract thinking (2). Analogous software implementations in artificial intelligence (e.g., deep learning networks) have proven effective in certain pattern classification tasks, such as speech and image recognition, but likewise have made little inroads in areas such as reasoning and natural language understanding. Is the search for a single canonical cortical circuit misguided?

[1]  R. von der Heydt,et al.  Synchrony and the binding problem in macaque visual cortex. , 2008, Journal of vision.

[2]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[3]  Andrew Y. Ng,et al.  Unsupervised learning models of primary cortical receptive fields and receptive field plasticity , 2011, NIPS.

[4]  Anthony M Zador,et al.  Differences in Sensitivity to Neural Timing among Cortical Areas , 2012, The Journal of Neuroscience.

[5]  E. Miller,et al.  Goal-direction and top-down control , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[6]  Terrence C. Stewart,et al.  of the Annual Meeting of the Cognitive Science Society Title Symbolic Reasoning in Spiking Neurons : A Model of the Cortex / Basal Ganglia / Thalamus Loop , 2010 .

[7]  Thomas K. Berger,et al.  Heterogeneity in the pyramidal network of the medial prefrontal cortex , 2006, Nature Neuroscience.

[8]  T. Hafting,et al.  Frequency of gamma oscillations routes flow of information in the hippocampus , 2009, Nature.

[9]  P. Dayan,et al.  Flexible shaping: How learning in small steps helps , 2009, Cognition.

[10]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[11]  Matthew Cook,et al.  Neuronal Projections Can Be Sharpened by a Biologically Plausible Learning Mechanism , 2011, ICANN.

[12]  R. Kurzweil How to Create a Mind: The Secret of Human Thought Revealed , 2012 .

[13]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[14]  J. Hawkins,et al.  On Intelligence , 2004 .

[15]  V. Mountcastle,et al.  An organizing principle for cerebral function : the unit module and the distributed system , 1978 .

[16]  Thomas C. Südhof,et al.  Cartography of neurexin alternative splicing mapped by single-molecule long-read mRNA sequencing , 2014, Proceedings of the National Academy of Sciences.

[17]  Guy N Elston,et al.  Cortical heterogeneity: Implications for visual processing and polysensory integration , 2002, Journal of neurocytology.

[18]  Richard G. Baraniuk,et al.  Sparse Coding via Thresholding and Local Competition in Neural Circuits , 2008, Neural Computation.

[19]  Lorenzo Rosasco,et al.  Unsupervised learning of invariant representations with low sample complexity: the magic of sensory cortex or a new framework for machine learning? , 2014 .

[20]  Kenneth J. Hayworth,et al.  Dynamically Partitionable Autoassociative Networks as a Solution to the Neural Binding Problem , 2012, Front. Comput. Neurosci..

[21]  J. R. Newton,et al.  Remodeling of Synaptic Structure in Sensory Cortical Areas In Vivo , 2006, The Journal of Neuroscience.

[22]  V. Steen,et al.  Gene expression in the rat brain: High similarity but unique differences between frontomedial-, temporal- and occipital cortex , 2011, BMC Neuroscience.

[23]  H. S. Meyer,et al.  Cellular organization of cortical barrel columns is whisker-specific , 2013, Proceedings of the National Academy of Sciences.

[24]  Wolfgang Maass,et al.  Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. , 2014, Cerebral cortex.

[25]  B. Dickson,et al.  Dscam diversity is essential for neuronal wiring and self-recognition , 2007, Nature.

[26]  Noam Chomsky,et al.  The faculty of language: what is it, who has it, and how did it evolve? , 2002, Science.

[27]  Earl K. Miller,et al.  Shifting the Spotlight of Attention: Evidence for Discrete Computations in Cognition , 2010, Front. Hum. Neurosci..

[28]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  C. Eliasmith,et al.  Higher-Dimensional Neurons Explain the Tuning and Dynamics of Working Memory Cells , 2006, The Journal of Neuroscience.

[30]  G. Elston Cortex, cognition and the cell: new insights into the pyramidal neuron and prefrontal function. , 2003, Cerebral cortex.

[31]  David Kappel,et al.  STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning , 2014, PLoS Comput. Biol..

[32]  P. Scheiffele,et al.  SAM68 Regulates Neuronal Activity-Dependent Alternative Splicing of Neurexin-1 , 2011, Cell.

[33]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[34]  Otto D. Creutzfeldt,et al.  Generality of the functional structure of the neocortex , 1977, Naturwissenschaften.

[35]  Alessandra Angelucci,et al.  Induction of visual orientation modules in auditory cortex , 2000, Nature.

[36]  T. Sejnowski,et al.  Correlated neuronal activity and the flow of neural information , 2001, Nature Reviews Neuroscience.

[37]  J. Sanes,et al.  Type II Cadherins Guide Assembly of a Direction-Selective Retinal Circuit , 2014, Cell.

[38]  Stephen J. Smith,et al.  Deep molecular diversity of mammalian synapses: why it matters and how to measure it , 2012, Nature Reviews Neuroscience.

[39]  Alexander Sher,et al.  Competition is a driving force in topographic mapping , 2011, Proceedings of the National Academy of Sciences.

[40]  S. Sherman The thalamus is more than just a relay , 2007, Current Opinion in Neurobiology.

[41]  Jonathan D. Cohen,et al.  Indirection and symbol-like processing in the prefrontal cortex and basal ganglia , 2013, Proceedings of the National Academy of Sciences.

[42]  C. Malsburg Binding in models of perception and brain function , 1995, Current Opinion in Neurobiology.

[43]  T Grant Belgard,et al.  Retooling spare parts: gene duplication and cognition , 2012, Nature Neuroscience.

[44]  Yuji Ikegaya,et al.  Synfire Chains and Cortical Songs: Temporal Modules of Cortical Activity , 2004, Science.

[45]  David J. Jilk,et al.  The Leabra architecture: Specialization without modularity , 2010, Behavioral and Brain Sciences.

[46]  Thomas C. Südhof,et al.  Presynaptic Neurexin-3 Alternative Splicing trans-Synaptically Controls Postsynaptic AMPA Receptor Trafficking , 2013, Cell.

[47]  Allan R. Jones,et al.  An anatomically comprehensive atlas of the adult human brain transcriptome , 2012, Nature.

[48]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[49]  Richard Hans Robert Hahnloser,et al.  Spike-Time-Dependent Plasticity and Heterosynaptic Competition Organize Networks to Produce Long Scale-Free Sequences of Neural Activity , 2010, Neuron.

[50]  T. Simpatico The Birth of the Mind: How a Tiny Number of Genes Creates the Complexities of Human Thought , 2005 .

[51]  C. Eliasmith,et al.  Learning to Select Actions with Spiking Neurons in the Basal Ganglia , 2012, Front. Neurosci..

[52]  Haim Sompolinsky,et al.  Nonlinear Population Codes , 2004, Neural Computation.

[53]  Friedemann Kiefer,et al.  Modulation of synaptic function through the α-neurexin–specific ligand neurexophilin-1 , 2014, Proceedings of the National Academy of Sciences.

[54]  E T Rolls,et al.  Computational constraints suggest the need for two distinct input systems to the hippocampal CA3 network , 1992, Hippocampus.

[55]  Jianing Yu,et al.  Top-down laminar organization of the excitatory network in motor cortex , 2008, Nature Neuroscience.

[56]  Henry Markram,et al.  Combinatorial Expression Rules of Ion Channel Genes in Juvenile Rat (Rattus norvegicus) Neocortical Neurons , 2012, PloS one.

[57]  M. Hasselmo,et al.  Free recall and recognition in a network model of the hippocampus: simulating effects of scopolamine on human memory function , 1997, Behavioural Brain Research.

[58]  S. Epstein,et al.  Gamma oscillations mediate stimulus competition and attentional selection in a cortical network model , 2008, Proceedings of the National Academy of Sciences.

[59]  G. Shepherd,et al.  Laminar and Columnar Organization of Ascending Excitatory Projections to Layer 2/3 Pyramidal Neurons in Rat Barrel Cortex , 2005, The Journal of Neuroscience.

[60]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[61]  Rajesh P. N. Rao,et al.  Probabilistic Models of the Brain: Perception and Neural Function , 2002 .

[62]  Nicholas A. Ketz,et al.  Thalamic pathways underlying prefrontal cortex–medial temporal lobe oscillatory interactions , 2015, Trends in Neurosciences.

[63]  D. Geschwind,et al.  Human-Specific Transcriptional Networks in the Brain , 2012, Neuron.

[64]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[65]  Sean L. Hill,et al.  Statistical connectivity provides a sufficient foundation for specific functional connectivity in neocortical neural microcircuits , 2012, Proceedings of the National Academy of Sciences.

[66]  Daniel L Adams,et al.  The cortical column: a structure without a function , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[67]  Rajesh P. N. Rao Decision Making Under Uncertainty: A Neural Model Based on Partially Observable Markov Decision Processes , 2010, Front. Comput. Neurosci..

[68]  G. Marcus The Algebraic Mind: Integrating Connectionism and Cognitive Science , 2001 .

[69]  D. Geschwind,et al.  Cortical Evolution: Judge the Brain by Its Cover , 2013, Neuron.

[70]  T DeWolf,et al.  The neural optimal control hierarchy for motor control , 2011, Journal of neural engineering.

[71]  Hongkui Zeng,et al.  Correlated gene expression and target specificity demonstrate excitatory projection neuron diversity. , 2015, Cerebral cortex.

[72]  Lorenzo Rosasco,et al.  The computational magic of the ventral stream: sketch of a theory (and why some deep architectures work). , 2012 .

[73]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[74]  D. Frost,et al.  Induction of functional retinal projections to the somatosensory system , 1985, Nature.

[75]  L.F. Abbott,et al.  Gating Multiple Signals through Detailed Balance of Excitation and Inhibition in Spiking Networks , 2009, Nature Neuroscience.

[76]  Geoffrey J. Goodhill,et al.  A simple model can unify a broad range of phenomena in retinotectal map development , 2011, Biological Cybernetics.

[77]  Matthew J. Traxler,et al.  What's Special About Human Language? The Contents of the "Narrow Language Faculty" Revisited , 2012, Lang. Linguistics Compass.

[78]  Chris Eliasmith,et al.  A Spiking Neuron Model of Serial-Order Recall , 2010 .

[79]  P. Ellen Grant,et al.  Evolutionarily Dynamic Alternative Splicing of GPR56 Regulates Regional Cerebral Cortical Patterning , 2014, Science.

[80]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[81]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[82]  Trevor Bekolay,et al.  Supplementary Materials for A Large-Scale Model of the Functioning Brain , 2012 .

[83]  H. Markram,et al.  The neocortical microcircuit as a tabula rasa. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[84]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[85]  R. O’Reilly Biologically Based Computational Models of High-Level Cognition , 2006, Science.

[86]  Stéphane Mallat Deep Learning by Scattering , 2013, ArXiv.

[87]  Tomaso A. Poggio,et al.  A Canonical Neural Circuit for Cortical Nonlinear Operations , 2008, Neural Computation.

[88]  Xiao-Jing Wang The Prefrontal Cortex as a Quintessential “Cognitive-Type” Neural Circuit , 2013 .

[89]  Thomas E. Hazy,et al.  Banishing the homunculus: Making working memory work , 2006, Neuroscience.

[90]  John R. Anderson How Can the Human Mind Occur in the Physical Universe , 2007 .

[91]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[92]  Nils Brose,et al.  The role of neurexins and neuroligins in the formation, maturation, and function of vertebrate synapses , 2012, Current Opinion in Neurobiology.

[93]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[94]  Robert F. Hadley The Problem of Rapid Variable Creation , 2009, Neural Computation.

[95]  Dimitri M. Kullmann,et al.  Oscillations and Filtering Networks Support Flexible Routing of Information , 2010, Neuron.

[96]  Peter Redgrave,et al.  A computational model of action selection in the basal ganglia. I. A new functional anatomy , 2001, Biological Cybernetics.

[97]  Charles R. Gallistel,et al.  Memory and the Computational Brain: Why Cognitive Science will Transform Neuroscience , 2009 .

[98]  Chris Eliasmith,et al.  How to Build a Brain: A Neural Architecture for Biological Cognition , 2013 .

[99]  A. Lansner,et al.  The cortex as a central pattern generator , 2005, Nature Reviews Neuroscience.

[100]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..