A method of automatically estimating the regularization parameter for Non-negative Matrix Factorization

The Idea of Non-negative Matrix Factorization (NMF) has been implemented in a wide variety of real world applications. To improve the usability of NMF, people usually add some regularization items to constrain the process of matrix factorization. The Regularized Non-negative Matrix Factorization (RNMF) mainly relies on prior knowledge to set the regularization parameter value. An improper regularization parameter value will directly influence the factorization result. Due to this reason, we propose a novel algorithm based on L-curve theory which is able to determine the regularization parameter value automatically, we name the algorithm as autoNMF. To testify the validity of autoNMF, we experiment the algorithm with the synthetic image data in blind source separation and compare it with other algorithms. Better unmixed results are gained indicating our algorithm outperforms other algorithms in several aspects.

[1]  Truong Q. Nguyen,et al.  A method for choosing the regularization parameter in generalized Tikhonov regularized linear inverse problems , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[2]  Per Christian Hansen,et al.  Regularization methods for large-scale problems , 1993 .

[3]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[5]  Andrzej Cichocki,et al.  Fast Nonnegative Matrix Factorization Algorithms Using Projected Gradient Approaches for Large-Scale Problems , 2008, Comput. Intell. Neurosci..

[6]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[7]  V. P. Pauca,et al.  Object Characterization from Spectral Data Using Nonnegative Factorization and Information Theory , 2004 .

[8]  Andrzej Cichocki,et al.  New Algorithms for Non-Negative Matrix Factorization in Applications to Blind Source Separation , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[9]  P. Paatero.  Least squares formulation of robust non-negative factor analysis , 1997 .

[10]  Sen Jia,et al.  Constrained Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[12]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[13]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[14]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[15]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[16]  Per Christian Hansen,et al.  Regularization Tools Version 3.0 for Matlab 5.2 , 1999, Numerical Algorithms.

[17]  Yin Zhang,et al.  Interior-Point Gradient Method for Large-Scale Totally Nonnegative Least Squares Problems , 2005 .

[18]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[19]  ProblemsPer Christian HansenDepartment.  The L-curve and its use in the numerical treatment of inverse problems , 2000 .

[20]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[21]  P. Green.  Bayesian reconstructions from emission tomography data using a modified EM algorithm. , 1990, IEEE transactions on medical imaging.

[22]  Jean-Francois Cardoso,et al.  Blind signal separation: statistical principles , 1998, Proc. IEEE.

[23]  H. Sebastian Seung,et al.  Unsupervised Learning by Convex and Conic Coding , 1996, NIPS.