Upper bound of Bayesian generalization error in non-negative matrix factorization

Non-negative matrix factorization (NMF) is a new knowledge discovery method that is used for text mining, signal processing, bioinformatics, and consumer analysis. However, its basic property as a learning machine is not yet clarified, as it is not a regular statistical model, resulting that theoretical optimization method of NMF has not yet established. In this paper, we study the real log canonical threshold of NMF and give an upper bound of the generalization error in Bayesian learning. The results show that the generalization error of the matrix factorization can be made smaller than regular statistical models if Bayesian learning is applied.

[1]  Haojie Li,et al.  Image multi-label annotation based on supervised nonnegative matrix factorization with new matching measurement , 2017, Neurocomputing.

[2]  L. Finesso,et al.  Nonnegative matrix factorization and I-divergence alternating minimization☆ , 2004, math/0412070.

[3]  Sumio Watanabe,et al.  Stochastic complexities of reduced rank regression in Bayesian estimation , 2005, Neural Networks.

[4]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[5]  Sumio Watanabe,et al.  Singularities in mixture models and upper bounds of stochastic complexity , 2003, Neural Networks.

[6]  Sumio Watanabe.  Algebraic geometrical methods for hierarchical learning machines , 2001, Neural Networks.

[7]  Sumio Watanabe,et al.  Algebraic geometry and stochastic complexity of hidden Markov models , 2005, Neurocomputing.

[8]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[9]  Hiroshi Sawada,et al.  Probabilistic Non-negative Inconsistent-resolution Matrices Factorization , 2015, CIKM.

[10]  M. Plummer,et al.  A Bayesian information criterion for singular models , 2013, 1309.0911.

[11]  Joel E. Cohen,et al.  Nonnegative ranks, decompositions, and factorizations of nonnegative matrices , 1993 .

[12]  Ali Taylan Cemgil,et al.  Bayesian Inference for Nonnegative Matrix Factorisation Models , 2009, Comput. Intell. Neurosci..

[13]  Yong He,et al.  A sparse unmixing model based on NMF and its application in Raman image , 2016, Neurocomputing.

[14]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[15]  Sumio Watanabe.  Algebraic Analysis for Non-regular Learning Machines , 1999, NIPS.

[16]  Nancy Bertin,et al.  Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis , 2009, Neural Computation.

[17]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .