Spatially correlated nonnegative matrix factorization

Low rank approximation of matrices has been frequently applied in information processing tasks, and in recent years, Nonnegative Matrix Factorization (NMF) has received considerable attentions for its straightforward interpretability and superior performance. When applied to image processing, ordinary NMF merely views a p"1xp"2 image as a vector in p"1xp"2-dimensional space and the pixels of the image are considered as independent. It fails to consider the fact that an image displayed in the plane is intrinsically a matrix, and pixels spatially close to each other may probably be correlated. Even though we have p"1xp"2 pixels per image, this spatial correlation suggests that the real number of freedom is far less. In this paper, we introduce a Spatially Correlated Nonnegative Matrix Factorization algorithm, which explicitly models the spatial correlation between neighboring pixels in the parts-based image representation. A multiplicative updating algorithm is also proposed to solve the corresponding optimization problem. Experimental results on benchmark image data sets demonstrate the effectiveness of the proposed method.

[1]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[2]  Tommi S. Jaakkola,et al.  Maximum-Margin Matrix Factorization , 2004, NIPS.

[3]  M.N.S. Swamy,et al.  Nonnegative Matrix Factorization , 2014 .

[4]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[5]  Yuxiao Hu,et al.  Learning a Spatially Smooth Subspace for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jian-Huang Lai,et al.  Extracting non-negative basis images using pixel dispersion penalty , 2012, Pattern Recognit..

[7]  Deng Cai,et al.  Tensor Subspace Analysis , 2005, NIPS.

[8]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[9]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[11]  Xiaofei He,et al.  Laplacian Regularized D-Optimal Design for Active Learning and Its Application to Image Retrieval , 2010, IEEE Transactions on Image Processing.

[12]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[13]  Karthik Devarajan,et al.  Nonnegative Matrix Factorization: An Analytical and Interpretive Tool in Computational Biology , 2008, PLoS Comput. Biol..

[14]  Andrzej Cichocki,et al.  Non-negative Matrix Factorization with Quasi-Newton Optimization , 2006, ICAISC.

[15]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[16]  Saeid Sanei,et al.  A new spatially constrained NMF with application to fMRI , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Jordi Vitrià,et al.  Non-negative Matrix Factorization for Face Recognition , 2002, CCIA.

[19]  Seung-Jean Kim,et al.  Hyperspectral Image Unmixing via Alternating Projected Subgradients , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[20]  Jiawei Han,et al.  Regularized locality preserving indexing via spectral regression , 2007, CIKM '07.

[21]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[22]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[23]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[24]  Daniel D. Lee,et al.  Multiplicative Updates for Nonnegative Quadratic Programming , 2007, Neural Computation.

[25]  F. O’Sullivan.  Discretized Laplacian Smoothing by Fourier Methods , 1991 .

[26]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[27]  Tao Li,et al.  The Relationships Among Various Nonnegative Matrix Factorization Methods for Clustering , 2006, Sixth International Conference on Data Mining (ICDM'06).

[28]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[29]  Demetri Terzopoulos,et al.  Multilinear subspace analysis of image ensembles , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Jiawei Han,et al.  Locally Consistent Concept Factorization for Document Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[31]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[32]  Wei-Ying Ma,et al.  Organizing WWW images based on the analysis of page layout and Web link structure , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[33]  Jidong Zhao,et al.  Locality sensitive semi-supervised feature selection , 2008, Neurocomputing.

[34]  Nicolas Gillis.  Nonnegative matrix factorization : complexity, algorithms and applications , 2011 .

[35]  Vishal Monga,et al.  Robust Image Hashing Via Non-Negative Matrix Factorizations , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[36]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[37]  Deng Cai,et al.  Orthogonal locality preserving indexing , 2005, SIGIR '05.

[38]  Tamir Hazan,et al.  Sparse image coding using a 3D non-negative tensor factorization , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[39]  Sen Jia,et al.  Constrained Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[40]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[41]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Nanning Zheng,et al.  Non-negative matrix factorization based methods for object recognition , 2004, Pattern Recognit. Lett..

[45]  Wei-Ying Ma,et al.  Clustering and searching WWW images using link and page layout analysis , 2004, TOMCCAP.

[46]  David L. Sheinberg,et al.  Visual object recognition. , 1996, Annual review of neuroscience.

[47]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[48]  Haesun Park,et al.  Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[49]  S. Sra.  Nonnegative Matrix Approximation: Algorithms and Applications , 2006 .

[50]  R. Plemmons,et al.  Optimality, computation, and interpretation of nonnegative matrix factorizations , 2004 .

[51]  J. Jost.  Riemannian geometry and geometric analysis , 1995 .

[52]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[53]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[54]  Hujun Bao,et al.  Understanding the Power of Clause Learning , 2009, IJCAI.

[55]  Levi Boyles,et al.  Statistical Optimization of Non-Negative Matrix Factorization , 2011, AISTATS.

[56]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .