Sequential EM learning for subspace analysis

Subspace analysis is one of popular multivariate data analysis methods, which has been widely used in pattern recognition. Typically data space belongs to very high dimension, but only a few principal components need to be extracted. In this paper, we present a fast sequential algorithm which behaves like expectation maximization (EM), for subspace analysis or tracking. In addition we also present a slight modification of the subspace algorithm by employing a rectifier, that is quite useful in handling nonnegative data (for example, images), which leads to rectified subspace analysis. The useful behavior of our proposed algorithms are confirmed through numerical experimental results with toy data and dynamic PET images.

[1]  Juha Karhunen,et al.  Principal component neural networks — Theory and applications , 1998, Pattern Analysis and Applications.

[2]  최승진 Non-negative matrix factorization of dynamic images in nuclear medicine , 2001 .

[3]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[4]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[5]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[6]  Zoubin Ghahramani,et al.  A Unifying Review of Linear Gaussian Models , 1999, Neural Computation.

[7]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[8]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[9]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[10]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[11]  D Charles,et al.  Modelling multiple-cause structure using rectification constraints. , 1998, Network.

[12]  Colin Fyfe,et al.  Simultaneous Identification of Face and Orientation , 2004, Neural Processing Letters.

[13]  Sam T. Roweis,et al.  EM Algorithms for PCA and SPCA , 1997, NIPS.

[14]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .

[15]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[16]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..