A Biological Gradient Descent for Prediction Through a Combination of STDP and Homeostatic Plasticity

Identifying, formalizing, and combining biological mechanisms that implement known brain functions, such as prediction, is a main aspect of research in theoretical neuroscience. In this letter, the mechanisms of spike-timing-dependent plasticity and homeostatic plasticity, combined in an original mathematical formalism, are shown to shape recurrent neural networks into predictors. Following a rigorous mathematical treatment, we prove that they implement the online gradient descent of a distance between the network activity and its stimuli. The convergence to an equilibrium, where the network can spontaneously reproduce or predict its stimuli, does not suffer from bifurcation issues usually encountered in learning in recurrent neural networks.

[1]  Prashant D. Sardeshmukh,et al.  The Optimal Growth of Tropical Sea Surface Temperature Anomalies , 1995 .

[2]  Moshe Bar,et al.  Predictions: a universal principle in the operation of the human brain , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[3]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[4]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[5]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[6]  이상헌,et al.  Deep Belief Networks , 2010, Encyclopedia of Machine Learning.

[7]  F. Verhulst,et al.  Averaging Methods in Nonlinear Dynamical Systems , 1985 .

[8]  Christos Dimitrakakis,et al.  Network Self-Organization Explains the Statistics and Dynamics of Synaptic Connection Strengths in Cortex , 2013, PLoS Comput. Biol..

[9]  Mathieu Galtier,et al.  A mathematical approach to unsupervised learning in recurrent neural networks. (Une approche mathématique de l'apprentissage non-supervisé dans les réseaux de neurones récurrents) , 2011 .

[10]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[11]  Gordon Pipa,et al.  2007 Special Issue: Fading memory and time series prediction in recurrent networks with different forms of plasticity , 2007 .

[12]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[13]  Barak A. Pearlmutter.  Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[14]  Roland Potthast,et al.  Inverse Problems in Neural Field Theory , 2009, SIAM J. Appl. Dyn. Syst..

[15]  Stefan J. Kiebel,et al.  Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks , 2012, Biological Cybernetics.

[16]  Rajesh P. N. Rao,et al.  Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning , 2001, Neural Computation.

[17]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[18]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[19]  D. Gilbert,et al.  Prospection: Experiencing the Future , 2007, Science.

[20]  T. Sejnowski.  Statistical constraints on synaptic plasticity. , 1977, Journal of theoretical biology.

[21]  D. Schacter,et al.  Episodic Simulation of Future Events , 2008, Annals of the New York Academy of Sciences.

[22]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[23]  Niraj S. Desai,et al.  Activity-dependent scaling of quantal amplitude in neocortical neurons , 1998, Nature.

[24]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[25]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[26]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[27]  K. Miller,et al.  Synaptic Economics: Competition and Cooperation in Synaptic Plasticity , 1996, Neuron.

[28]  Bradley E. Alger,et al.  An Improved Test for Detecting Multiplicative Homeostatic Synaptic Scaling , 2012, PloS one.

[29]  A. Grinvald,et al.  Spontaneously emerging cortical representations of visual attributes , 2003, Nature.

[30]  Wulfram Gerstner,et al.  Spike-timing dependent plasticity , 2010, Scholarpedia.

[31]  R. F. Galán,et al.  On How Network Architecture Determines the Dominant Patterns of Spontaneous Neural Activity , 2008, PLoS ONE.

[32]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[33]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[34]  Wulfram Gerstner,et al.  Spiking Neuron Models: Single Neurons, Populations, Plasticity , 2002 .

[35]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction , 2001 .

[36]  Gordon Pipa,et al.  SORN: A Self-Organizing Recurrent Neural Network , 2009, Front. Comput. Neurosci..

[37]  Kenneth D. Miller,et al.  The Role of Constraints in Hebbian Learning , 1994, Neural Computation.

[38]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[39]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[40]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[41]  Gilles Wainrib,et al.  Multiscale analysis of slow-fast neuronal learning models with noise , 2012, Journal of mathematical neuroscience.

[42]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[43]  Wulfram Gerstner,et al.  Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns , 1993, Biological Cybernetics.

[44]  Laurenz Wiskott,et al.  Slowness: An Objective for Spike-Timing–Dependent Plasticity? , 2007, PLoS Comput. Biol..

[45]  Wulfram Gerstner,et al.  Spiking Neuron Models: An Introduction , 2002 .

[46]  Jesper Tegnér,et al.  Reverse engineering gene networks using singular value decomposition and robust regression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[47]  Herbert Jaeger,et al.  Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes , 2014, Neural Networks.

[48]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[49]  Jean-Pascal Pfister,et al.  STDP in Adaptive Neurons Gives Close-To-Optimal Information Transmission , 2010, Front. Comput. Neurosci..

[50]  Eugene M. Izhikevich,et al.  Relating STDP to BCM , 2003, Neural Computation.

[51]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[52]  A. Clark.  Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[53]  Mark C. W. van Rossum,et al.  Stable Hebbian Learning from Spike Timing-Dependent Plasticity , 2000, The Journal of Neuroscience.

[54]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[55]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[56]  Silvia Scarpetta,et al.  Spatiotemporal learning in analog neural networks using spike-timing-dependent synaptic plasticity. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[57]  S. Nelson,et al.  Homeostatic plasticity in the developing nervous system , 2004, Nature Reviews Neuroscience.

[58]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[59]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[60]  K. Doya,et al.  Bifurcations in the learning of recurrent neural networks , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[61]  Sophie Denève,et al.  Bayesian Spiking Neurons I: Inference , 2008, Neural Computation.

[62]  G. Turrigiano.  Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same , 1999, Trends in Neurosciences.

[63]  Karl J. Friston.  The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.