Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm

The error backpropagation learning algorithm (BP) is generally considered biologically implausible because it does not use locally available, activation-based variables. A version of BP that can be computed locally using bidirectional activation recirculation (Hinton and McClelland 1988) instead of backpropagated error derivatives is more biologically plausible. This paper presents a generalized version of the recirculation algorithm (GeneRec), which overcomes several limitations of the earlier algorithm by using a generic recurrent network with sigmoidal units that can learn arbitrary input/output mappings. However, the contrastive Hebbian learning algorithm (CHL, also known as DBM or mean field learning) also uses local variables to perform error-driven learning in a sigmoidal recurrent network. CHL was derived in a stochastic framework (the Boltzmann machine), but has been extended to the deterministic case in various ways, all of which rely on problematic approximations and assumptions, leading some to conclude that it is fundamentally flawed. This paper shows that CHL can be derived instead from within the BP framework via the GeneRec algorithm. CHL is a symmetry-preserving version of GeneRec that uses a simple approximation to the midpoint or second-order accurate Runge-Kutta method of numerical integration, which explains the generally faster learning speed of CHL compared to BI. Thus, all known fully general error-driven learning algorithms that use local activation-based variables in deterministic networks can be considered variations of the GeneRec algorithm (and indirectly, of the backpropagation algorithm). GeneRec therefore provides a promising framework for thinking about how the brain might perform error-driven learning. To further this goal, an explicit biological mechanism is proposed that would be capable of implementing GeneRec-style learning. This mechanism is consistent with available evidence regarding synaptic modification in neurons in the neocortex and hippocampus, and makes further predictions.

[1]  J. Haldane.  The interaction of nature and nurture. , 1946, Annals of eugenics.

[2]  D. Whitteridge,et al.  Learning and Relearning , 1959, Science's STKE.

[3]  E. John,et al.  Evoked-Potential Correlates of Stimulus Uncertainty , 1965, Science.

[4]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  Francis Crick,et al.  The function of dream sleep , 1983, Nature.

[6]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[10]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[11]  Carsten Peterson,et al.  A Mean Field Theory Learning Algorithm for Neural Networks , 1987, Complex Syst..

[12]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[13]  Geoffrey E. Hinton,et al.  Learning Representations by Recirculation , 1987, NIPS.

[14]  Fernando J. Pineda,et al.  Generalization of Back propagation to Recurrent and Higher Order Neural Networks , 1987, NIPS.

[15]  Fernando J. Pineda,et al.  GENERALIZATION OF BACKPROPAGATION TO RECURRENT AND HIGH-ORDER NETWORKS. , 1987 .

[16]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[17]  T. Bliss,et al.  NMDA receptors - their role in long-term potentiation , 1987, Trends in Neurosciences.

[18]  Fernando J. Pineda,et al.  Dynamics and architecture for neural computation , 1988, J. Complex..

[19]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[20]  Geoffrey E. Hinton.  Deterministic Boltzmann Learning Performs Steepest Descent in Weight-Space , 1989, Neural Computation.

[21]  Geoffrey E. Hinton.  Learning distributed representations of concepts. , 1989 .

[22]  Carsten Peterson,et al.  Explorations of the mean field theory learning algorithm , 1989, Neural Networks.

[23]  Francis Crick,et al.  The recent excitement about neural networks , 1989, Nature.

[24]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[25]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[26]  J. Lisman,et al.  A mechanism for the Hebb and the anti-Hebb processes underlying learning and memory. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Geoffrey E. Hinton,et al.  Discovering High Order Features with Mean Field Modules , 1989, NIPS.

[28]  Gerald Tesauro,et al.  Neural models of classical conditioning: A theoretical viewpoint. , 1990 .

[29]  L. B. Almeida.  A learning rule for asynchronous perceptrons with feedback in a combinatorial environment , 1990 .

[30]  W. Singer,et al.  Different voltage-dependent thresholds for inducing long-term depression and long-term potentiation in slices of rat visual cortex , 1990, Nature.

[31]  Michael I. Jordan,et al.  A more biologically plausible learning rule for neural networks. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[32]  William A. Phillips,et al.  A Biologically Supported Error-Correcting Learning Rule , 1991, Neural Computation.

[33]  Carsten Peterson,et al.  Mean Field Theory Neural Networks for Feature Recognition, Content Addressable Memory and Optimization , 1991 .

[34]  Javier R. Movellan,et al.  Contrastive Hebbian Learning in the Continuous Hopfield Model , 1991 .

[35]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[36]  Geoffrey E. Hinton,et al.  Deterministic Boltzmann Learning in Networks with Asymmetric Connectivity , 1991 .

[37]  Ralph Linsker,et al.  Local Synaptic Learning Rules Suffice to Maximize Mutual Information in a Linear Network , 1992, Neural Computation.

[38]  W. N. Ross,et al.  The spread of Na+ spikes determines the pattern of dendritic Ca2+ entry into hippocampal neurons , 1992, Nature.

[39]  W Singer,et al.  Intracellular injection of Ca2+ chelators blocks induction of long-term depression in rat visual cortex. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Dimitri M. Kullmann,et al.  Ca2+ Entry via postsynaptic voltage-sensitive Ca2+ channels can transiently potentiate excitatory synaptic transmission in the hippocampus , 1992, Neuron.

[41]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[42]  F. Crépel,et al.  Postsynaptic calcium is necessary for the induction of LTP and LTD of monosynaptic EPSPs in prefrontal neurons: An in vitro study in the rat , 1992, Synapse.

[43]  R. Malenka,et al.  Temporal limits on the rise in postsynaptic calcium required for the induction of long-term potentiation , 1992, Neuron.

[44]  R. Malenka,et al.  Mechanisms underlying induction of homosynaptic long-term depression in area CA1 of the hippocampus , 1992, Neuron.

[45]  W. Singer,et al.  Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation , 1993, Trends in Neurosciences.

[46]  C. Galland.  The limitations of deterministic Boltzmann machine learning , 1993 .

[47]  R. Nicoll,et al.  NMDA-receptor-dependent synaptic plasticity: multiple forms and mechanisms , 1993, Trends in Neurosciences.

[48]  J. B. Levitt,et al.  Topography of pyramidal neuron intrinsic connections in macaque monkey prefrontal cortex (areas 9 and 46) , 1993, The Journal of comparative neurology.

[49]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[50]  David Zipser,et al.  The neurobiological significance of the new learning models , 1993 .

[51]  G. Collingridge,et al.  Induction of LTP in the hippocampus needs synaptic activation of glutamate metabotropic receptors , 1993, Nature.

[52]  R. Nicoll,et al.  The role of Ca2+ entry via synaptically activated NMDA receptors in the induction of long-term potentiation , 1993, Neuron.

[53]  James L. McClelland,et al.  A Parallel Distributed Processing Perspective , 1994 .

[54]  J. Lisman.  The CaM kinase II hypothesis for the storage of synaptic memory , 1994, Trends in Neurosciences.

[55]  M. Bear,et al.  Synaptic plasticity: LTP and LTD , 1994, Current Opinion in Neurobiology.

[56]  D. Linden,et al.  Long-term synaptic depression in the mammalian brain , 1994, Neuron.