The generative Adaptive Subspace Self-Organizing Map

The Adaptive Subspace Self Organized Map (ASSOM) is a model that incorporates sparsity, nonlinear pooling, topological organization and temporal continuity to learn invariant feature detectors, each corresponding to one node of the network. Temporal continuity is implemented by grouping inputs into "training episodes". Each episode contains samples from one invariance class and is mapped to a particular node during training. However, this explicit grouping makes application of this algorithm for natural image sequences difficult, since the grouping is generally not known a priori. This work proposes a probabilistic generative model of the ASSOM that addresses this problem. Each node of the ASSOM generates input vectors from one invariance class. Training sequences are generated by nodes that are chosen according to a Markov process. We demonstrate that this model can learn invariant feature detectors similar to those found in the primary visual cortex from an unlabeled sequence of input images generated by a realistic model of eye movements. Performance is comparable to the original ASSOM algorithm, but without the need for explicit grouping into training episodes.

[1]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[2]  Samuel Kaski,et al.  Self-Organized Formation of Various Invariant-Feature Filters in the Adaptive-Subspace SOM , 1997, Neural Computation.

[3]  Olivier Capp'e.  Online EM Algorithm for Hidden Markov Models , 2009, 0908.2359.

[4]  A. Grinvald,et al.  The layout of iso-orientation domains in area 18 of cat visual cortex: optical imaging reveals a pinwheel-like organization , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  J. Victor,et al.  Temporal Encoding of Spatial Information during Active Visual Fixation , 2012, Current Biology.

[8]  Samuel Kaski,et al.  The Adaptive-Subspace Self-Organizing Map (ASSOM) , 1997 .

[9]  Laurenz Wiskott,et al.  Slow feature analysis yields a rich repertoire of complex cell properties. , 2005, Journal of vision.

[10]  Eric Moulines,et al.  On‐line expectation–maximization algorithm for latent data models , 2007, ArXiv.

[11]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[12]  Alex Bateman,et al.  An introduction to hidden Markov models. , 2007, Current protocols in bioinformatics.

[13]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[14]  R. Fergus,et al.  Learning invariant features through topographic filter maps , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[16]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[17]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[18]  Amiram Grinvald,et al.  Iso-orientation domains in cat visual cortex are arranged in pinwheel-like patterns , 1991, Nature.

[19]  Teuvo Kohonen,et al.  Emergence of invariant-feature detectors in the adaptive-subspace self-organizing map , 1996, Biological Cybernetics.

[20]  Huicheng Zheng,et al.  Fast-Learning Adaptive-Subspace Self-Organizing Map: An Application to Saliency-Based Invariant Image Feature Construction , 2008, IEEE Transactions on Neural Networks.

[21]  O. Cappé,et al.  On‐line expectation–maximization algorithm for latent data models , 2009 .

[22]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[23]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[24]  M. Rolfs.  Microsaccades: Small steps on a long way , 2009, Vision Research.

[25]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.