Biological learning curves outperform existing ones in artificial intelligence algorithms

Recently, deep learning algorithms have outperformed human experts in various tasks across several domains; however, their characteristics are distant from current knowledge of neuroscience. The simulation results of biological learning algorithms presented herein outperform state-of-the-art optimal learning curves in supervised learning of feedforward networks. The biological learning algorithms comprise asynchronous input signals with decaying input summation, weights adaptation, and multiple outputs for an input signal. In particular, the generalization error for such biological perceptrons decreases rapidly with increasing number of examples, and it is independent of the size of the input. This is achieved using either synaptic learning, or solely through dendritic adaptation with a mechanism of swinging between reflecting boundaries, without learning steps. The proposed biological learning algorithms outperform the optimal scaling of the learning curve in a traditional perceptron. It also results in a considerable robustness to disparity between weights of two networks with very similar outputs in biological supervised learning scenarios. The simulation results indicate the potency of neurobiological mechanisms and open opportunities for developing a superior class of deep learning algorithms.

[1]  Bo Li,et al.  Exploring the Function Space of Deep-Learning Machines , 2017, Physical review letters.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[4]  Konrad P. Körding,et al.  Toward an Integration of Deep Learning and Neuroscience , 2016, bioRxiv.

[5]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[6]  Wolfgang Kinzel,et al.  Improving a Network Generalization Ability by Selecting Examples , 1990 .

[7]  Sarah Webb.  Deep learning for biology , 2018, Nature.

[8]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[9]  N. Spruston.  Pyramidal neurons: dendritic structure and synaptic integration , 2008, Nature Reviews Neuroscience.

[10]  Jonghee Yoon,et al.  Holographic deep learning for rapid optical screening of anthrax spores , 2017, Science Advances.

[11]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[12]  Opper,et al.  Generalization performance of Bayes optimal classification algorithm for learning a perceptron. , 1991, Physical review letters.

[13]  Opper,et al.  Mean field approach to Bayes learning in feed-forward neural networks. , 1996, Physical review letters.

[14]  Wolfgang Kinzel,et al.  Freezing transition in asymmetric random neural networks with deterministic dynamics , 1989 .

[15]  M. Timme,et al.  Stable irregular dynamics in complex neural networks. , 2007, Physical review letters.

[16]  Daniel W. Davies,et al.  Machine learning for molecular and materials science , 2018, Nature.

[17]  H. Risken.  Fokker-Planck Equation , 1984 .

[18]  Ido Kanter,et al.  Adaptive nodes enrich nonlinear cooperative learning beyond traditional adaptation by links , 2018, Scientific Reports.

[19]  W. Gerstner,et al.  Spike-Timing-Dependent Plasticity: A Comprehensive Overview , 2012, Front. Syn. Neurosci..

[20]  Marc Timme,et al.  Unstable attractors induce perpetual synchronization and desynchronization. , 2002, Chaos.

[21]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[22]  Marc Timme,et al.  Statistical physics of neural systems with non-additive dendritic coupling , 2015, 1507.03881.

[23]  Shane Legg,et al.  Universal Intelligence: A Definition of Machine Intelligence , 2007, Minds and Machines.

[24]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[25]  Srdjan Ostojic,et al.  Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons , 2014, Nature Neuroscience.

[26]  T. Watkin,et al.  THE STATISTICAL-MECHANICS OF LEARNING A RULE , 1993 .

[27]  Dimitri M. Kullmann,et al.  Oscillatory multiplexing of population codes for selective communication in the mammalian brain , 2014, Nature Reviews Neuroscience.

[28]  M. Bethge,et al.  Common input explains higher-order correlations and entropy in a simple model of neural population activity. , 2011, Physical review letters.

[29]  Adriano Barra,et al.  On the equivalence of Hopfield networks and Boltzmann Machines , 2011, Neural Networks.

[30]  Konrad P. Kording,et al.  Towards an integration of deep learning and neuroscience , 2016, bioRxiv.

[31]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[32]  Michael Biehl,et al.  Learning by on-line gradient descent , 1995 .

[33]  G. Laurent,et al.  Conditional modulation of spike-timing-dependent plasticity for olfactory learning , 2012, Nature.

[34]  P. Dayan,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S9 References the Asynchronous State in Cortical Circuits , 2022 .

[35]  W. Kinzel,et al.  Strong and weak chaos in nonlinear networks with time-delayed couplings. , 2011, Physical review letters.

[36]  Florian Marquardt,et al.  Reinforcement Learning with Neural Networks for Quantum Feedback , 2018, Physical Review X.

[37]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[38]  O. Kinouchi,et al.  Optimal generalization in perceptions , 1992 .

[39]  Danica Kragic,et al.  From active perception to deep learning , 2018, Science Robotics.

[40]  Florent Krzakala,et al.  A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines , 2017, Physical Review X.

[41]  H. Risken.  Fokker-Planck Equation , 1996 .

[42]  Youngjin Park,et al.  Symmetry of learning rate in synaptic plasticity modulates formation of flexible and stable memories , 2017, Scientific Reports.

[43]  Elena Agliari,et al.  Hierarchical neural networks perform both serial and parallel processing , 2014, Neural Networks.

[44]  Adriano Barra,et al.  A new mechanical approach to handle generalized Hopfield neural networks , 2018, Neural Networks.

[45]  Wolfgang Kinzel,et al.  Mutual learning in a tree parity machine and its application to cryptography. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.