Learning the parts of objects by auto-association

Recognition-by-components is one of the possible strategies proposed for object recognition by the brain, but little is known about the low-level mechanism by which the parts of objects can be learned without a priori knowledge. Recent work by Lee and Seung (Nature 401 (1999) 788) shows the importance of non-negativity constraints in the building of such models. Here we propose a simple feedforward neural network that is able to learn the parts of objects by the auto-association of sensory stimuli. The network is trained to reproduce each input with only excitatory interactions. When applied to a database of facial images, the network extracts localized features that resemble intuitive notion of the parts of faces. This kind of localized, parts-based internal representation is very different from the holistic representation created by the unconstrained network, which emulates principal component analysis. Furthermore, the simple model has some ability to minimize the number of active hidden units for certain tasks and is robust when a mixture of different stimuli is presented.

[1]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[2]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[3]  Linda Dychkowski,et al.  Character building. , 2002, School nurse news.

[4]  P. Paatero,et al.  Investigation of sources of atmospheric aerosol at urban and suburban residential areas in Thailand by positive matrix factorization , 2000 .

[5]  P. Paatero.  Least squares formulation of robust non-negative factor analysis , 1997 .

[6]  D Zipser,et al.  Learning the hidden structure of speech. , 1988, The Journal of the Acoustical Society of America.

[7]  Giovanni Soda,et al.  Autoassociator-based models for speaker verification , 1996, Pattern Recognit. Lett..

[8]  M. Gluck,et al.  Hippocampal mediation of stimulus representation: A computational theory , 1993, Hippocampus.

[9]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[10]  葛 錫金.  Extracting knowledge from databases by redundancy reduction , 2000 .

[11]  Jun S. Huang,et al.  Separating similar complex Chinese characters by Walsh transform , 1987, Pattern Recognit..

[12]  R Hecht-Nielsen,et al.  Replicator neural networks for universal optimal source coding. , 1995, Science.

[13]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[14]  Dominique Valentin,et al.  CAN A LINEAR AUTOASSOCIATOR RECOGNIZE FACES FROM NEW ORIENTATIONS , 1996 .

[15]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[16]  A. Martínez,et al.  The AR face databasae , 1998 .

[17]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[18]  N. E. Sharkey,et al.  Models of cognition : a review of cognitive science , 1989 .

[19]  Alice J. O'Toole,et al.  Connectionist models of face processing: A survey , 1994, Pattern Recognit..

[20]  M. Arozullah,et al.  Image compression with a hierarchical neural network , 1996, IEEE Transactions on Aerospace and Electronic Systems.

[21]  Nathalie Japkowicz,et al.  Nonlinear Autoassociation Is Not Equivalent to PCA , 2000, Neural Computation.

[22]  Jian Feng,et al.  Entropy illustrates the flexibility of Chinese , 2001, Nature.

[23]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[24]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[25]  Peter Dayan,et al.  A simple algorithm that discovers efficient perceptual codes , 1997 .

[26]  Garrison W. Cottrell,et al.  Image compression by back-propagation: An example of extensional programming , 1988 .

[27]  Laurence R. Harris,et al.  Computational and psychophysical mechanisms of visual coding , 1997 .

[28]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[29]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[30]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[31]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[32]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .