Perceptron Learning and Classification in a Modeled Cortical Pyramidal Cell

The perceptron learning algorithm and its multiple-layer extension, the backpropagation algorithm, are the foundations of the present-day machine learning revolution. However, these algorithms utilize a highly simplified mathematical abstraction of a neuron; it is not clear to what extent real biophysical neurons with morphologically-extended nonlinear dendritic trees and conductance-based synapses could realize perceptron-like learning. Here we implemented the perceptron learning algorithm in a realistic biophysical model of a layer 5 cortical pyramidal cell. We tested this biophysical perceptron (BP) on a memorization task, where it needs to correctly binarily classify 100, 1000, or 2000 patterns, and a generalization task, where it should discriminate between two “noisy” patterns. We show that the BP performs these tasks with an accuracy comparable to that of the original perceptron, though the memorization capacity of the apical tuft is somewhat limited. We concluded that cortical pyramidal neurons can act as powerful classification devices.

[1]  J. Schiller,et al.  NMDA spikes in basal dendrites of cortical pyramidal neurons , 2000, Nature.

[2]  Bertalan K. Andrásfalvy,et al.  Location-dependent synaptic plasticity rules by dendritic spine cooperativity , 2016, Nature Communications.

[3]  Clifton C. Rumsey,et al.  Synaptic democracy in active dendrites. , 2006, Journal of neurophysiology.

[4]  D. Amit,et al.  Perceptron learning with sign-constrained weights , 1989 .

[5]  Konrad P. Körding,et al.  Integrating Top-Down and Bottom-Up Sensory Processing by Somato-Dendritic Interactions , 2004, Journal of Computational Neuroscience.

[6]  Idan Segev,et al.  Single cortical neurons as deep artificial neural networks , 2019, Neuron.

[7]  H. Sompolinsky,et al.  Time-Warp–Invariant Neuronal Processing , 2009, PLoS biology.

[8]  M. Larkum A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex , 2013, Trends in Neurosciences.

[9]  N. Spruston,et al.  Determinants of Voltage Attenuation in Neocortical Pyramidal Neuron Dendrites , 1998, The Journal of Neuroscience.

[10]  Bartlett W. Mel,et al.  Synaptic plasticity in dendrites: complications and coping strategies , 2017, Current Opinion in Neurobiology.

[11]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.

[12]  Y. Dan,et al.  Activity Recall in Visual Cortical Ensemble , 2012, Nature Neuroscience.

[13]  S. Siegelbaum,et al.  Dendritic Na+ spikes enable cortical input to drive action potential output from hippocampal CA2 pyramidal neurons , 2014, eLife.

[14]  Razvan V. Florian,et al.  The Chronotron: A Neuron That Learns to Fire Temporally Precise Spike Patterns , 2010, PloS one.

[15]  Robert Gütig,et al.  Spiking neurons can discover predictive features by aggregate-label learning , 2016, Science.

[16]  E. Newport,et al.  Computation of Conditional Probability Statistics by 8-Month-Old Infants , 1998 .

[17]  Stephen D Van Hooser,et al.  Laminar organization of response properties in primary visual cortex of the gray squirrel (Sciurus carolinensis). , 2005, Journal of neurophysiology.

[18]  J. Budd Extrastriate feedback to primary visual cortex in primates: a quantitative analysis of connectivity , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[19]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[20]  K. Svoboda,et al.  Long-term in vivo imaging of experience-dependent synaptic plasticity in adult cortex , 2002, Nature.

[21]  Johannes J. Letzkus,et al.  Requirement of dendritic calcium spikes for induction of spike‐timing‐dependent synaptic plasticity , 2006, The Journal of physiology.

[22]  Nicolas Brunel,et al.  A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks , 2015, PLoS Comput. Biol..

[23]  Michael W. Spratling Cortical region interactions and the functional role of apical dendrites. , 2002, Behavioral and cognitive neuroscience reviews.

[24]  Bartlett W. Mel,et al.  Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural Tissue , 2001, Neuron.

[25]  C. Koch,et al.  Sparse but not ‘Grandmother-cell’ coding in the medial temporal lobe , 2008, Trends in Cognitive Sciences.

[26]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Paul A. Rhodes,et al.  The Properties and Implications of NMDA Spikes in Neocortical Pyramidal Cells , 2006, The Journal of Neuroscience.

[29]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[30]  J. Nadal,et al.  Optimal Information Storage and the Distribution of Synaptic Weights Perceptron versus Purkinje Cell , 2004, Neuron.

[31]  Julio Chapeton,et al.  Efficient associative memory storage in cortical circuits of inhibitory and excitatory neurons , 2012, Proceedings of the National Academy of Sciences.

[32]  Urit Gordon,et al.  Plasticity Compartments in Basal Dendrites of Neocortical Pyramidal Neurons , 2006, The Journal of Neuroscience.

[33]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[34]  Bartlett W. Mel The Clusteron: Toward a Simple Abstraction for a Complex Neuron , 1991, NIPS.

[35]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[36]  Idan Segev,et al.  Spike-timing-dependent synaptic plasticity and synaptic democracy in dendrites. , 2009, Journal of neurophysiology.

[37]  N. Spruston Pyramidal neurons: dendritic structure and synaptic integration , 2008, Nature Reviews Neuroscience.

[38]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[39]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[40]  F. H. C. Crick,et al.  Certain aspects of the anatomy and physiology of the cerebral cortex , 1986 .

[41]  Patrick M. Pilarski,et al.  First steps towards an intelligent laser welding architecture using deep neural networks and reinforcement learning , 2014 .

[42]  J. Albus A Theory of Cerebellar Function , 1971 .

[43]  Neil Davey,et al.  Nonspecific synaptic plasticity improves the recognition of sparse patterns degraded by local noise , 2017, Scientific Reports.

[44]  Judit K. Makara,et al.  Compartmentalized dendritic plasticity and input feature storage in neurons , 2008, Nature.

[45]  N. Brunel,et al.  Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location , 2012, Proceedings of the National Academy of Sciences.

[46]  Sten Grillner,et al.  Input specificity and dependence of spike timing-dependent plasticity on preceding postsynaptic activity at unitary connections between neocortical layer 2/3 pyramidal cells. , 2009, Cerebral cortex.

[47]  W. Rall Distinguishing theoretical synaptic potentials computed for different soma-dendritic distributions of synaptic input. , 1967, Journal of neurophysiology.

[48]  S. Manita,et al.  A Top-Down Cortical Circuit for Accurate Sensory Perception , 2015, Neuron.

[49]  Guy Eyal,et al.  Human Cortical Pyramidal Neurons: From Spines to Spikes via Models , 2018, bioRxiv.

[50]  Idan Segev,et al.  Principles Governing the Operation of Synaptic Inhibition in Dendrites , 2012, Neuron.

[51]  W. Senn,et al.  Learning by the Dendritic Prediction of Somatic Spiking , 2014, Neuron.

[52]  Bartlett W. Mel,et al.  Pyramidal Neuron as Two-Layer Neural Network , 2003, Neuron.

[53]  Idan Segev,et al.  Single Cortical Neurons as Deep Artificial Neural Networks , 2019 .

[54]  Keith B. Hengen,et al.  Firing Rate Homeostasis in Visual Cortex of Freely Behaving Rodents , 2013, Neuron.

[55]  W. Gan,et al.  Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity , 2015, Nature.

[56]  Matthew E Larkum,et al.  Synaptic clustering by dendritic signalling mechanisms , 2008, Current Opinion in Neurobiology.

[57]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[58]  E. Maire,et al.  The Influence of Light Stimulation on Subcortical Potentials Evoked by Single Flashes in Photosensitive Papio papio , 1986, Epilepsia.

[59]  D. Kullmann,et al.  Plasticity of Inhibition , 2012, Neuron.

[60]  Tobias Bonhoeffer,et al.  Precision of Inhibition: Dendritic Inhibition by Individual GABAergic Synapses on Hippocampal Pyramidal Cells Is Confined in Space and Time , 2015, Neuron.

[61]  Nace L. Golding,et al.  Dendritic Sodium Spikes Are Variable Triggers of Axonal Action Potentials in Hippocampal CA1 Pyramidal Neurons , 1998, Neuron.

[62]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[63]  Bartlett W. Mel,et al.  Computational subunits in thin dendrites of pyramidal cells , 2004, Nature Neuroscience.

[64]  Henry Markram,et al.  Timed Synaptic Inhibition Shapes NMDA Spikes, Influencing Local Dendritic Processing and Global I/O Properties of Cortical Neurons. , 2017, Cell reports.

[65]  D. Johnston,et al.  Characterization of single voltage‐gated Na+ and Ca2+ channels in apical dendrites of rat CA1 pyramidal neurons. , 1995, The Journal of physiology.

[66]  Idan Segev,et al.  Perceptron Learning and Classification in a Modeled Cortical Pyramidal Cell , 2020, Frontiers in Computational Neuroscience.

[67]  M. Häusser,et al.  Synaptic function: Dendritic democracy , 2001, Current Biology.

[68]  Alcino J. Silva,et al.  Synaptic clustering within dendrites: An emerging theory of memory formation , 2015, Progress in Neurobiology.

[69]  William Wisden,et al.  Synaptic inhibition of Purkinje cells mediates consolidation of vestibulo-cerebellar motor learning , 2009, Nature Neuroscience.

[70]  Odelia Schwartz,et al.  Statistical Learning of Melodic Patterns Influences the Brain's Response to Wrong Notes , 2017, Journal of Cognitive Neuroscience.

[71]  Michael L. Hines,et al.  Neuroinformatics Original Research Article Neuron and Python , 2022 .

[72]  Henry Markram,et al.  Models of Neocortical Layer 5b Pyramidal Cells Capturing a Wide Range of Dendritic and Perisomatic Active Properties , 2011, PLoS Comput. Biol..

[73]  Walter Senn,et al.  Somato-dendritic Synaptic Plasticity and Error-backpropagation in Active Dendrites , 2016, PLoS Comput. Biol..

[74]  Idan Segev,et al.  Modeling a layer 4-to-layer 2/3 module of a single column in rat neocortex: Interweaving in vitro and in vivo experimental observations , 2007, Proceedings of the National Academy of Sciences.

[75]  Bartlett W. Mel NMDA-Based Pattern Discrimination in a Modeled Cortical Neuron , 1992, Neural Computation.

[76]  T. Freund,et al.  Total number and distribution of inhibitory and excitatory synapses on hippocampal CA1 pyramidal cells , 2001, Neuroscience.