Automatic Rank Determination in Projective Nonnegative Matrix Factorization

Projective Nonnegative Matrix Factorization (PNMF) has demonstrated advantages in both sparse feature extraction and clustering. However, PNMF requires users to specify the column rank of the approximative projection matrix, the value of which is unknown before-hand. In this paper, we propose a method called ARDPNMF to automatically determine the column rank in PNMF. Our method is based on automatic relevance determination (ARD) with Jeffrey's prior. After deriving the multiplicative update rule using the expectation-maximization technique for ARDPNMF, we test it on various synthetic and real-world datasets for feature extraction and clustering applications to show the effectiveness of our algorithm. For FERET faces and the Swimmer dataset, interpretable number of features are obtained correctly via our algorithm. Several UCI datasets for clustering are also tested, in which we find that ARDPNMF can estimate the number of clusters quite accurately with low deviation and good cluster purity.

[1]  Erkki Oja,et al.  Linear and Nonlinear Projective Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[2]  Inderjit S. Dhillon,et al.  Generalized Nonnegative Matrix Approximations with Bregman Divergences , 2005, NIPS.

[3]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[4]  Leszek Wojnar,et al.  Image Analysis , 1998 .

[5]  Jorma Laaksonen,et al.  Projective Non-Negative Matrix Factorization with Applications to Facial Image Processing , 2007, Int. J. Pattern Recognit. Artif. Intell..

[6]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[7]  Christopher M. Bishop,et al.  Bayesian PCA , 1998, NIPS.

[8]  Mário A. T. Figueiredo.  Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[11]  Erkki Oja,et al.  Projective Nonnegative Matrix Factorization for Image Compression and Feature Extraction , 2005, SCIA.

[12]  David Mackay,et al.  Probable networks and plausible predictions - a review of practical Bayesian methods for supervised neural networks , 1995 .

[13]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  C. Févotte,et al.  Automatic Relevance Determination in Nonnegative Matrix Factorization , 2009 .

[15]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[16]  Seungjin Choi,et al.  Algorithms for orthogonal nonnegative matrix factorization , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).