Single cortical neurons as deep artificial neural networks

Utilizing recent advances in machine learning, we introduce a systematic approach to characterize neurons' input/output (I/O) mapping complexity. Deep neural networks (DNNs) were trained to faithfully replicate the I/O function of various biophysical models of cortical neurons at millisecond (spiking) resolution. A temporally convolutional DNN with five to eight layers was required to capture the I/O mapping of a realistic model of a layer 5 cortical pyramidal cell (L5PC). This DNN generalized well when presented with inputs widely outside the training distribution. When NMDA receptors were removed, a much simpler network (fully connected neural network with one hidden layer) was sufficient to fit the model. Analysis of the DNNs' weight matrices revealed that synaptic integration in dendritic branches could be conceptualized as pattern matching from a set of spatiotemporal templates. This study provides a unified characterization of the computational complexity of single neurons and suggests that cortical networks therefore have a unique architecture, potentially supporting their computational power.

[1]  J. Schiller,et al.  NMDA spikes in basal dendrites of cortical pyramidal neurons , 2000, Nature.

[2]  M. Larkum,et al.  Dendritic action potentials and computation in human layer 2/3 cortical neurons , 2020, Science.

[3]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[4]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[5]  Subutai Ahmad,et al.  Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex , 2015, Front. Neural Circuits.

[6]  Idan Segev,et al.  Signal enhancement in distal cortical dendrites by means of interactions between active dendritic spines. , 1985, Proceedings of the National Academy of Sciences of the United States of America.

[7]  C. Stevens,et al.  Calcium permeability of the N-methyl-D-aspartate receptor channel in hippocampal neurons in culture. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[8]  James G. King,et al.  Reconstruction and Simulation of Neocortical Microcircuitry , 2015, Cell.

[9]  Derek Nowrouzezahrai,et al.  Subspace neural physics: fast data-driven interactive simulation , 2019, Symposium on Computer Animation.

[10]  W. Rall.  Branching dendritic trees and motoneuron membrane resistivity. , 1959, Experimental neurology.

[11]  Surya Ganguli,et al.  A deep learning framework for neuroscience , 2019, Nature Neuroscience.

[12]  N. Spruston,et al.  Synapse Distribution Suggests a Two-Stage Model of Dendritic Integration in CA1 Pyramidal Neurons , 2009, Neuron.

[13]  M. Häusser,et al.  Dendritic Discrimination of Temporal Input Sequences in Cortical Neurons , 2010, Science.

[14]  B. Sakmann,et al.  Calcium action potentials restricted to distal apical dendrites of rat neocortical pyramidal neurons , 1997, The Journal of physiology.

[15]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[16]  Noa Rogozinski,et al.  An efficient analytical reduction of detailed nonlinear neuron models , 2020, Nature Communications.

[17]  Boris S. Gutkin,et al.  Passive Dendrites Enable Single Neurons to Compute Linearly Non-separable Functions , 2013, PLoS Comput. Biol..

[18]  Walter Senn,et al.  Data-driven reduction of dendritic morphologies with preserved dendro-somatic responses , 2020, bioRxiv.

[19]  Bartlett W. Mel,et al.  Dendrites: bug or feature? , 2003, Current Opinion in Neurobiology.

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  J. Schiller,et al.  Active properties of neocortical pyramidal neuron dendrites. , 2013, Annual review of neuroscience.

[22]  Christof Koch,et al.  The role of single neurons in information processing , 2000, Nature Neuroscience.

[23]  Alexander Mordvintsev,et al.  Inceptionism: Going Deeper into Neural Networks , 2015 .

[24]  N. Spruston,et al.  Activity-dependent action potential invasion and calcium influx into hippocampal CA1 dendrites. , 1995, Science.

[25]  Bartlett W. Mel,et al.  Pyramidal Neuron as Two-Layer Neural Network , 2003, Neuron.

[26]  Panayiota Poirazi,et al.  Challenging the point neuron dogma: FS basket cells as 2-stage nonlinear integrators , 2018, Nature Communications.

[27]  Idan Segev,et al.  The role of dendritic inhibition in shaping the plasticity of excitatory synapses , 2013, Front. Neural Circuits.

[28]  B. Sakmann,et al.  Active propagation of somatic action potentials into neocortical pyramidal cell dendrites , 1994, Nature.

[29]  Bartlett W. Mel,et al.  Mechanisms underlying subunit independence in pyramidal neuron dendrites , 2013, Proceedings of the National Academy of Sciences.

[30]  W Rall,et al.  Computational study of an excitable dendritic spine. , 1988, Journal of neurophysiology.

[31]  Judit K. Makara,et al.  Compartmentalized dendritic plasticity and input feature storage in neurons , 2008, Nature.

[32]  Judit K. Makara,et al.  Global and Multiplexed Dendritic Computations under In Vivo-like Conditions , 2018, Neuron.

[33]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[34]  Bartlett W. Mel.  NMDA-Based Pattern Discrimination in a Modeled Cortical Neuron , 1992, Neural Computation.

[35]  A. Polsky,et al.  Synaptic Integration in Tuft Dendrites of Layer 5 Pyramidal Neurons: A New Unifying Principle , 2009, Science.

[36]  N. Spruston,et al.  Action potential initiation and backpropagation in neurons of the mammalian CNS , 1997, Trends in Neurosciences.

[37]  Gretchen A. Stevens,et al.  Heterogeneous contributions of change in population distribution of body mass index to change in obesity and underweight , 2020, eLife.

[38]  Idan Segev,et al.  The Impact of Parallel Fiber Background Activity on the Cable Properties of Cerebellar Purkinje Cells , 1992, Neural Computation.

[39]  W. Rall.  Distinguishing theoretical synaptic potentials computed for different soma-dendritic distributions of synaptic input. , 1967, Journal of neurophysiology.

[40]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[41]  Henry Markram,et al.  Models of Neocortical Layer 5b Pyramidal Cells Capturing a Wide Range of Dendritic and Perisomatic Active Properties , 2011, PLoS Comput. Biol..

[42]  E. Viezzer,et al.  Up to two billion times acceleration of scientific simulations with deep neural architecture search , 2020, ArXiv.

[43]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Idan Segev,et al.  Perceptron Learning and Classification in a Modeled Cortical Pyramidal Cell , 2018, bioRxiv.

[45]  B. Sakmann,et al.  A new cellular mechanism for coupling inputs arriving at different cortical layers , 1999, Nature.

[46]  Henry Markram,et al.  Timed Synaptic Inhibition Shapes NMDA Spikes, Influencing Local Dendritic Processing and Global I/O Properties of Cortical Neurons. , 2017, Cell reports.

[47]  Wulfram Gerstner,et al.  The quantitative single-neuron modeling competition , 2008, Biological Cybernetics.

[48]  Demis Hassabis,et al.  Improved protein structure prediction using potentials from deep learning , 2020, Nature.

[49]  S. Strogatz.  Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering , 1995 .

[50]  Leon A. Gatys,et al.  Deep convolutional models improve predictions of macaque V1 responses to natural images , 2017, bioRxiv.

[51]  Anthony M. Zador,et al.  Nonlinear Pattern Separation in Single Hippocampal Neurons with Active Dendritic Membrane , 1991, NIPS.

[52]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  D. Johnston,et al.  Characterization of single voltage‐gated Na+ and Ca2+ channels in apical dendrites of rat CA1 pyramidal neurons. , 1995, The Journal of physiology.

[54]  Alcino J. Silva,et al.  Synaptic clustering within dendrites: An emerging theory of memory formation , 2015, Progress in Neurobiology.

[55]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[56]  Bartlett W. Mel,et al.  Arithmetic of Subthreshold Synaptic Summation in a Model CA1 Pyramidal Cell , 2003, Neuron.

[57]  Anthony N. Burkitt,et al.  A Review of the Integrate-and-fire Neuron Model: I. Homogeneous Synaptic Input , 2006, Biological Cybernetics.

[58]  Hans-Christian Hege,et al.  Generation of dense statistical connectomes from sparse morphological data , 2014, Front. Neuroanat..

[59]  Bartlett W. Mel,et al.  Computational subunits in thin dendrites of pyramidal cells , 2004, Nature Neuroscience.

[60]  R. Reid,et al.  Predicting Every Spike A Model for the Responses of Visual Neurons , 2001, Neuron.

[61]  T. Poggio,et al.  Retinal ganglion cells: a functional interpretation of dendritic morphology. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[62]  Wulfram Gerstner,et al.  Spike-timing prediction in cortical neurons with active dendrites , 2014, Front. Comput. Neurosci..