Symmetric Predictive Estimator for Biologically Plausible Neural Learning

In a real brain, the act of perception is a bidirectional process, depending on both feedforward sensory pathways and feedback pathways that carry expectations. We are interested in how such a neural network might emerge from a biologically plausible learning rule. Other neural network learning methods either only apply to feedforward networks, or employ assumptions (such as weight copying) that render them unlikely in a real brain. Predictive estimators (PEs) offer a better solution to this bidirectional learning scenario. However, PEs also depend on weight copying. In this paper, we propose the symmetric PE (SPE), an architecture that can learn both feedforward and feedback connection weights individually using only locally available information. We demonstrate that the SPE can learn complicated mappings without the use of weight copying. The SPE networks also show promise in deeper architectures.

[1]  James J. Gibson,et al.  The Ecological Approach to Visual Perception: Classic Edition , 2014 .

[2]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[3]  Anil K. Jain,et al.  Artificial Neural Networks: A Tutorial , 1996, Computer.

[4]  Trevor Bekolay,et al.  Nengo: a Python tool for building large-scale functional brain models , 2014, Front. Neuroinform..

[5]  S. Sherman,et al.  Synaptic Properties of Corticocortical Connections between the Primary and Secondary Visual Cortical Areas in the Mouse , 2011, The Journal of Neuroscience.

[6]  Trevor Bekolay,et al.  Simultaneous unsupervised and supervised learning of cognitive functions in biologically plausible spiking neural networks , 2013, CogSci.

[7]  Yoshua Bengio,et al.  Towards Biologically Plausible Deep Learning , 2015, ArXiv.

[8]  Stephen Grossberg,et al.  Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world , 2013, Neural Networks.

[9]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[10]  A. Borst.  Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[11]  Bradley S Peterson,et al.  Deficits in Predictive Coding Underlie Hallucinations in Schizophrenia , 2014, The Journal of Neuroscience.

[12]  Dimitri P. Bertsekas,et al.  A New Class of Incremental Gradient Methods for Least Squares Problems , 1997, SIAM J. Optim..

[13]  G. Henry,et al.  Physiological studies on the feedback connection to the striate cortex from cortical areas 18 and 19 of the cat , 1988, Experimental Brain Research.

[14]  Daniel Cownden,et al.  Random feedback weights support learning in deep neural networks , 2014, ArXiv.

[15]  Jeroen J. A. van Boxtel,et al.  A predictive coding perspective on autism spectrum disorders , 2013, Front. Psychology.

[16]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[17]  M. Mignard,et al.  Paths of information flow through visual cortex. , 1991, Science.

[18]  G. Rizzolatti,et al.  Understanding motor events: a neurophysiological study , 2004, Experimental Brain Research.

[19]  Karl J. Friston,et al.  Predictive coding: an account of the mirror neuron system , 2007, Cognitive Processing.

[20]  L. A. N. Esq.,et al.  LXI. Observations on some remarkable optical phænomena seen in Switzerland; and on an optical phænomenon which occurs on viewing a figure of a crystal or geometrical solid , 1832 .

[21]  C. Gilbert,et al.  Generation of end-inhibition in the visual cortex via interlaminar connections , 1986, Nature.

[22]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[23]  David Xu.  Biologically Plausible Neural Learning using Symmetric Predictive Estimators , 2016 .

[24]  J. O’Keefe,et al.  Phase relationship between hippocampal place units and the EEG theta rhythm , 1993, Hippocampus.

[25]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[26]  Joel Z. Leibo,et al.  How Important Is Weight Symmetry in Backpropagation? , 2015, AAAI.

[27]  BART KOSKO,et al.  Bidirectional associative memories , 1988, IEEE Trans. Syst. Man Cybern..

[28]  R. Hecht-Nielsen.  Counterpropagation networks. , 1987, Applied optics.

[29]  S. Grossberg,et al.  Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors , 1976, Biological Cybernetics.

[30]  Chris Eliasmith,et al.  Fine-Tuning and the Stability of Recurrent Neural Networks , 2011, PloS one.

[31]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[32]  Pierre Baldi,et al.  A theory of local learning, the learning channel, and the optimality of backpropagation , 2015, Neural Networks.

[33]  Yoshua Bengio,et al.  Difference Target Propagation , 2014, ECML/PKDD.

[34]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.