A Metric for Evaluating Neural Input Representation in Supervised Learning Networks

Supervised learning has long been attributed to several feed-forward neural circuits within the brain, with particular attention being paid to the cerebellar granular layer. The focus of this study is to evaluate the input activity representation of these feed-forward neural networks. The activity of cerebellar granule cells is conveyed by parallel fibers and translated into Purkinje cell activity, which constitutes the sole output of the cerebellar cortex. The learning process at this parallel-fiber-to-Purkinje-cell connection makes each Purkinje cell sensitive to a set of specific cerebellar states, which are roughly determined by the granule-cell activity during a certain time window. A Purkinje cell becomes sensitive to each neural input state and, consequently, the network operates as a function able to generate a desired output for each provided input by means of supervised learning. However, not all sets of Purkinje cell responses can be assigned to any set of input states due to the network's own limitations (inherent to the network neurobiological substrate), that is, not all input-output mapping can be learned. A key limiting factor is the representation of the input states through granule-cell activity. The quality of this representation (e.g., in terms of heterogeneity) will determine the capacity of the network to learn a varied set of outputs. Assessing the quality of this representation is interesting when developing and studying models of these networks to identify those neuron or network characteristics that enhance this representation. In this study we present an algorithm for evaluating quantitatively the level of compatibility/interference amongst a set of given cerebellar states according to their representation (granule-cell activation patterns) without the need for actually conducting simulations and network training. The algorithm input consists of a real-number matrix that codifies the activity level of every considered granule-cell in each state. The capability of this representation to generate a varied set of outputs is evaluated geometrically, thus resulting in a real number that assesses the goodness of the representation.

[1]  Mehran Kardar,et al.  Information optimization in coupled audio-visual cortical maps , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[2]  E. Knudsen Supervised learning in the brain , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  Mitsuo Kawato,et al.  Cerebellar supervised learning revisited: biophysical modeling and degrees-of-freedom control , 2011, Current Opinion in Neurobiology.

[4]  Claudia Clopath,et al.  Sparse synaptic connectivity is required for decorrelation and pattern separation in feedforward networks , 2017, Nature Communications.

[5]  E. D’Angelo,et al.  Theta-Frequency Resonance at the Cerebellum Input Stage Improves Spike Timing on the Millisecond Time-Scale , 2013, Front. Neural Circuits.

[6]  Eduardo Ros,et al.  From Sensors to Spikes: Evolving Receptive Fields to Enhance Sensorimotor Information in a Robot-Arm , 2012, Int. J. Neural Syst..

[7]  Hermann Wagner,et al.  The representation of sound localization cues in the barn owl's inferior colliculus , 2012, Front. Neural Circuits.

[8]  Konstantin Avrachenkov,et al.  The Multi-Dimensional Version of ∫ba xp dx , 2001, Am. Math. Mon..

[9]  Morteza Esmaeili,et al.  Extension of the Lasserre-Avrachenkov theorem on the integral of multilinear forms over simplices , 2009, Appl. Math. Comput..

[10]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[11]  J. Leo van Hemmen,et al.  Inhibition, not excitation, is the key to multimodal sensory integration , 2008, Biological Cybernetics.

[12]  Jianfeng Feng,et al.  Computational neuroscience , 1986, Behavioral and Brain Sciences.

[13]  Conor J. Houghton,et al.  Parameter estimation of neuron models using in-vitro and in-vivo electrophysiological data , 2015, Front. Neuroinform..

[14]  T. Bliss,et al.  Plasticity in the human central nervous system. , 2006, Brain : a journal of neurology.

[15]  F. A. Miles,et al.  Long-term adaptive changes in primate vestibuloocular reflex. IV. Electrophysiological observations in flocculus of adapted monkeys. , 1980, Journal of neurophysiology.

[16]  Arne Møller,et al.  Total numbers of various cell types in rat cerebellar cortex estimated using an unbiased stereological method , 1993, Brain Research.

[17]  Egidio D'Angelo,et al.  Distributed cerebellar plasticity implements adaptable gain control in a manipulation task: a closed-loop robotic simulation , 2013, Front. Neural Circuits.

[18]  Alcino J. Silva,et al.  Molecular mechanisms of synaptic plasticity and memory , 1999, Current Opinion in Neurobiology.

[19]  D. Marr A theory of cerebellar cortex , 1969, The Journal of physiology.

[20]  A. Sideris,et al.  Learning convergence in the cerebellar model articulation controller , 1992, IEEE Trans. Neural Networks.

[21]  Daniele Mortari,et al.  The n-dimensional cross product and its application to the matrix eigenanalysis , 1996 .

[22]  Y. Dan,et al.  Spike Timing-Dependent Plasticity of Neural Circuits , 2004, Neuron.

[23]  Perry L. Miller,et al.  Twenty years of ModelDB and beyond: building essential modeling tools for the future of neuroscience , 2016, Journal of Computational Neuroscience.

[24]  D. Avis On the Extreme Rays of the Metric Cone , 1980, Canadian Journal of Mathematics.

[25]  Eduardo Ros,et al.  Cerebellar Input Configuration Toward Object Model Abstraction in Manipulation Tasks , 2011, IEEE Transactions on Neural Networks.

[26]  John Porrill,et al.  Provided for Non-commercial Research and Educational Use Only. Not for Reproduction, Distribution or Commercial Use. Decorrelation Learning in the Cerebellum: Computational Analysis and Experimental Questions 7 Author's Personal Copy , 2022 .

[27]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[28]  Henry Markram,et al.  BluePyOpt: Leveraging Open Source Software and Cloud Infrastructure to Optimise Model Parameters in Neuroscience , 2016, Front. Neuroinform..

[29]  Eduardo Ros,et al.  A real-time spiking cerebellum model for learning robot control , 2008, Biosyst..

[30]  E. Knudsen Instructed learning in the auditory localization pathway of the barn owl , 2002, Nature.

[31]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[32]  Jesús A. De Loera,et al.  Software for exact integration of polynomials over polyhedra , 2011, ACCA.

[33]  Stefano Fusi,et al.  The Sparseness of Mixed Selectivity Neurons Controls the Generalization–Discrimination Trade-Off , 2013, The Journal of Neuroscience.

[34]  Felix Schürmann,et al.  Single Neuron Optimization as a Basis for Accurate Biophysical Modeling: The Case of Cerebellar Granule Cells , 2017, Front. Cell. Neurosci..

[35]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[36]  T Tyrrell,et al.  Cerebellar cortex: its simulation and the relevance of Marr's theory. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[37]  M. Kawato,et al.  The cerebellum and VOR/OKR learning models , 1992, Trends in Neurosciences.

[38]  David H. Eberly,et al.  Geometric Tools for Computer Graphics , 2002 .

[39]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[40]  Pablo Martínez-Cañada,et al.  A Computational Framework for Realistic Retina Modeling , 2016, Int. J. Neural Syst..

[41]  Melvyn W. Jeter Mathematical Programming: An Introduction to Optimization , 1986 .

[42]  Eduardo Ros,et al.  Cerebellarlike Corrective Model Inference Engine for Manipulation Tasks , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  S. Lisberger,et al.  Neural Learning Rules for the Vestibulo-Ocular Reflex , 1998, The Journal of Neuroscience.

[44]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[45]  D. G. Clayton,et al.  Gram‐Schmidt Orthogonalization , 1971 .

[46]  M. Fujita,et al.  Adaptive filter model of the cerebellum , 1982, Biological Cybernetics.

[47]  Daniele Mortari,et al.  N-DIMENSIONAL CROSS PRODUCT AND ITS APPLICATION TO MATRIX EIGENANALYSIS , 1997 .

[48]  Nicolas Brunel,et al.  A Cerebellar Learning Model of Vestibulo-Ocular Reflex Adaptation in Wild-Type and Mutant Mice , 2014, The Journal of Neuroscience.

[49]  Silvia Tolu,et al.  Event-driven simulation of cerebellar granule cells , 2008, Biosyst..

[50]  K. Doya,et al.  Unsupervised learning of granule cell sparse codes enhances cerebellar adaptive control , 2001, Neuroscience.

[51]  Eduardo Ros,et al.  Event-driven simulation of neural population synchronization facilitated by electrical coupling , 2007, Biosyst..

[52]  Donghui Chen,et al.  Nonnegativity constraints in numerical analysis , 2009, The Birth of Numerical Analysis.

[53]  M Rucci,et al.  Registration of Neural Maps through Value-Dependent Learning: Modeling the Alignment of Auditory and Visual Maps in the Barn Owl’s Optic Tectum , 1997, The Journal of Neuroscience.

[54]  F A Miles,et al.  Long-term adaptive changes in primate vestibuloocular reflex. I. Behavioral observations. , 1980, Journal of neurophysiology.

[55]  Andreas K Engel,et al.  Auditory and visual interactions between the superior and inferior colliculi in the ferret , 2015, The European journal of neuroscience.

[56]  L. Abbott,et al.  Synaptic computation , 2004, Nature.

[57]  Silvia Tolu,et al.  Oscillation-Driven Spike-Timing Dependent Plasticity Allows Multiple Overlapping Pattern Recognition in Inhibitory Interneuron Networks , 2016, Int. J. Neural Syst..

[58]  Thierry Nieus,et al.  A Realistic Large-Scale Model of the Cerebellum Granular Layer Predicts Circuit Spatio-Temporal Filtering Properties , 2009, Front. Cell. Neurosci..

[59]  K. Doya Complementary roles of basal ganglia and cerebellum in learning and motor control , 2000, Current Opinion in Neurobiology.

[60]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[61]  Terry T. Takahashi,et al.  How the owl tracks its prey – II , 2010, Journal of Experimental Biology.

[62]  J. Albus A Theory of Cerebellar Function , 1971 .

[63]  I. Fujita,et al.  Distribution of GABAergic neurons and terminals in the auditory system of the barn owl , 1989, The Journal of comparative neurology.

[64]  Jean-François Richard,et al.  Methods of Numerical Integration , 2000 .

[65]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[66]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[67]  Nathaniel B Sawtell,et al.  Multimodal Integration in Granule Cells as a Basis for Associative Plasticity and Sensory Prediction in a Cerebellum-like Circuit , 2010, Neuron.

[68]  M. Häusser,et al.  Tonic Synaptic Inhibition Modulates Neuronal Output Pattern and Spatiotemporal Synaptic Integration , 1997, Neuron.

[69]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[70]  C. I. De Zeeuw,et al.  Timing in the cerebellum: oscillations and resonance in the granular layer , 2009, Neuroscience.