Upper Bound of Bayesian Generalization Error in Stochastic Matrix Factorization

Stochastic matrix factorization (SMF) has proposed and it can be understood as a restriction to non-negative matrix factorization (NMF). SMF is useful for inference of topic models, NMF for binary matrices data, and Bayesian Network. However, it needs some strong assumption to reach unique factorization in SMF and also theoretical prediction accuracy has not yet clarified. In this paper, we study the maximum pole of zeta function of SMF and derive an upper bound of the generalization error in Bayesian inference. This results give the foundation of establishing widely applicable and rigorous factorization method for SMF and mean that the generalization error in SMF can become smaller than regular statistical models by Bayesian inference.

[1]  Sumio Watanabe.  Algebraic Analysis for Non-regular Learning Machines , 1999, NIPS.

[2]  Sumio Watanabe.  Algebraic geometrical methods for hierarchical learning machines , 2001, Neural Networks.

[3]  Sumio Watanabe,et al.  Singularities in mixture models and upper bounds of stochastic complexity , 2003, Neural Networks.

[4]  Sumio Watanabe,et al.  Stochastic Complexity of Bayesian Networks , 2003, UAI.

[5]  Sumio Watanabe,et al.  Algebraic geometry and stochastic complexity of hidden Markov models , 2005, Neurocomputing.

[6]  Sumio Watanabe,et al.  Stochastic complexities of reduced rank regression in Bayesian estimation , 2005, Neural Networks.

[7]  Sumio Watanabe,et al.  Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory , 2010, J. Mach. Learn. Res..

[8]  M. Plummer,et al.  A Bayesian information criterion for singular models , 2013, 1309.0911.

[9]  Hiroshi Sawada,et al.  Probabilistic Non-negative Inconsistent-resolution Matrices Factorization , 2015, CIKM.

[10]  Line Harder Clemmensen,et al.  Non-negative Matrix Factorization for binary data , 2015, 2015 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K).

[11]  Christopher Adams,et al.  Finite Mixture Models with One Exclusion Restriction , 2016 .

[12]  Christopher Adams,et al.  Stochastic Matrix Factorization , 2016, ArXiv.

[13]  Naoki Hayashi,et al.  Upper bound of Bayesian generalization error in non-negative matrix factorization , 2016, Neurocomputing.