Self-Organization of Topographic Bilinear Networks for Invariant Recognition

We present a model for the emergence of ordered fiber projections that may serve as a basis for invariant recognition. After invariance transformations are self-organized, so-called control units competitively activate fiber projections for different transformation parameters. The model builds on a well-known ontogenetic mechanism, activity-based development of retinotopy, and it employs activity blobs of varying position and size to install different transformations. We provide a detailed analysis for the case of 1D input and output fields for schematic input patterns that shows how the model is able to develop specific mappings. We discuss results that show that the proposed learning scheme is stable for complex, biologically more realistic input patterns. Finally, we show that the model generalizes to 2D neuronal fields driven by simulated retinal waves.

[1]  Jerome A. Feldman,et al.  Dynamic connections in neural networks , 1990, Biological Cybernetics.

[2]  Bruno A. Olshausen,et al.  Bilinear models of natural images , 2007, Electronic Imaging.

[3]  Rajesh P. N. Rao,et al.  Bilinear Sparse Coding for Invariant Vision , 2005, Neural Computation.

[4]  Junmei Zhu.  Synaptic Formation Rate as a Control Parameter in a Model for the Ontogenesis of Retinotopy , 2008, ICANN.

[5]  J. Matthias Starck,et al.  Avian growth and development : evolution within the altricial-precocial spectrum , 1998 .

[6]  G. Goodhill.  Contributions of Theoretical Modeling to the Understanding of Neural Map Development , 2007, Neuron.

[7]  M. Feller,et al.  Mechanisms underlying development of visual maps and receptive fields. , 2008, Annual review of neuroscience.

[8]  Christoph von der Malsburg,et al.  Maplets for correspondence-based object recognition , 2004, Neural Networks.

[9]  D. Baylor,et al.  Synchronous bursts of action potentials in ganglion cells of the developing mammalian retina. , 1991, Science.

[10]  Christoph von der Malsburg,et al.  A Bilinear Model for Consistent Topographic Representations , 2010, ICANN.

[11]  Geoffrey E. Hinton.  A Parallel Computation that Assigns Canonical Object-Based Frames of Reference , 1981, IJCAI.

[12]  Christian Wolff,et al.  Dynamic Link Matching between Feature Columns for Different Scale and Orientation , 2007, ICONIP.

[13]  Niraj S. Desai,et al.  Plasticity in the intrinsic excitability of cortical pyramidal neurons , 1999, Nature Neuroscience.

[14]  Jörg Lücke,et al.  Glial cells for information routing? , 2007, Cognitive Systems Research.

[15]  Christoph von der Malsburg,et al.  Self-Organization of Steerable Topographic Mappings as Basis for Translation Invariance , 2010, ICANN.

[16]  Jörg Lücke,et al.  Hierarchical self-organization of minicolumnar receptive fields , 2004, Neural Networks.

[17]  J. Rothman,et al.  Synaptic depression enables neuronal gain control , 2009, Nature.

[18]  Christoph von der Malsburg,et al.  What Is the Optimal Architecture for Visual Information Routing? , 2007, Neural Computation.

[19]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[20]  R. Khazipov,et al.  GABA: a pioneer transmitter that excites immature neurons and generates primitive oscillations. , 2007, Physiological reviews.

[21]  N. Swindale.  The development of topography in the visual cortex: a review of models. , 1996, Network.

[22]  E. KostShelton,et al.  Growth and development. , 1951, Ciba clinical symposia.

[23]  Richard E. Turner,et al.  A Structured Model of Video Reproduces Primary Visual Cortical Organisation , 2009, PLoS Comput. Biol..

[24]  A. Hendrickson,et al.  Primate foveal development: a microcosm of current questions in neurobiology. , 1994, Investigative ophthalmology & visual science.

[25]  Terrence J. Sejnowski,et al.  Constrained Optimization for Neural Map Formation: A Unifying Framework for Weight Growth and Normalization , 1998, Neural Computation.

[26]  Jörg Lücke,et al.  Rapid Convergence to Feature Layer Correspondences , 2008, Neural Computation.

[27]  M. Eigen.  Selforganization of matter and the evolution of biological macromolecules , 1971, Naturwissenschaften.

[28]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[29]  C. von der Malsburg.  Self-organization of orientation sensitive cells in the striate cortex. , 1973, Kybernetik.

[30]  Joshua B. Tenenbaum,et al.  Separating Style and Content with Bilinear Models , 2000, Neural Computation.

[31]  Tomoki Fukai,et al.  A Simple Neural Network Exhibiting Selective Activation of Neuronal Ensembles: From Winner-Take-All to Winners-Share-All , 1997, Neural Computation.

[32]  L. Abbott,et al.  A model of multiplicative neural responses in parietal cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Christian Wolff,et al.  A recurrent dynamic model for correspondence-based face recognition. , 2008, Journal of vision.

[34]  Alexander Sher,et al.  Spatial-Temporal Patterns of Retinal Waves Underlying Activity-Dependent Refinement of Retinofugal Projections , 2009, Neuron.

[35]  Christoph von der Malsburg,et al.  Recognizing Faces by Dynamic Link Matching , 1996, NeuroImage.

[36]  Aapo Hyvärinen,et al.  Topographic Independent Component Analysis , 2001, Neural Computation.

[37]  Geoffrey E. Hinton,et al.  Learning to Represent Spatial Transformations with Factored Higher-Order Boltzmann Machines , 2010, Neural Computation.

[38]  D C Van Essen,et al.  Shifter circuits: a computational strategy for dynamic aspects of visual processing. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[39]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[40]  Mark H. Johnson,et al.  CONSPEC and CONLERN: a two-process theory of infant face recognition. , 1991, Psychological review.

[41]  David W. Arathorn,et al.  Map-Seeking Circuits in Visual Cognition: A Computational Mechanism for Biological and Machine Vision , 2002 .

[42]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[43]  Holger G. Krapp,et al.  Multiplication and stimulus invariance in a looming-sensitive neuron , 2004, Journal of Physiology-Paris.

[44]  F. Simion,et al.  Can a Nonspecific Bias Toward Top-Heavy Patterns Explain Newborns' Face Preference? , 2004, Psychological science.

[45]  D. Linden,et al.  The other side of the engram: experience-driven changes in neuronal intrinsic excitability , 2003, Nature Reviews Neuroscience.

[46]  C. Chiu,et al.  Spontaneous Activity in Developing Ferret Visual Cortex In Vivo , 2001, The Journal of Neuroscience.

[47]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[48]  Joel L. Davis,et al.  An Introduction to Neural and Electronic Networks , 1995 .

[49]  H. A.F.,et al.  DEVELOPMENT OF RETINOTOPIC PROJECTIONS: AN ANALYTIC TREATMENT , 1983 .

[50]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[51]  C. C. Goren,et al.  Visual following and pattern discrimination of face-like stimuli by newborn infants. , 1975, Pediatrics.

[52]  Nicholas V. Swindale,et al.  Retinal Wave Behavior through Activity-Dependent Refractory Periods , 2007, PLoS Comput. Biol..

[53]  Andrew D Huberman,et al.  Dynamics of Spontaneous Activity in the Fetal Macaque Retina during Development of Retinogeniculate Pathways , 2006, The Journal of Neuroscience.