On Rationality of Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is the problem of decomposing a given nonnegative n × m matrix M into a product of a nonnegative n × d matrix W and a nonnegative d × m matrix H. NMF has a wide variety of applications, including bioinformatics, chemometrics, communication complexity, machine learning, polyhedral combinatorics, among many others. A longstanding open question, posed by Cohen and Rothblum in 1993, is whether every rational matrix M has an NMF with minimal d whose factors W and H are also rational. We answer this question negatively, by exhibiting a matrix M for which W and H require irrational entries. As an application of this result, we show that state minimization of labeled Markov chains can require the introduction of irrational transition probabilities. We complement these irrationality results with an NP-complete version of NMF for which rational numbers suffice.

[1]  Michael Sipser,et al.  Inference and minimization of hidden Markov chains , 1994, COLT '94.

[2]  Christel Baier,et al.  Probabilistic ω-automata , 2012, JACM.

[3]  Naoki Abe,et al.  On the computational complexity of approximating distributions by probabilistic automata , 1990, Machine Learning.

[4]  James Worrell,et al.  On Restricted Nonnegative Matrix Factorization , 2016, ICALP.

[5]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[6]  Sanjeev Arora,et al.  Learning Topic Models -- Going beyond SVD , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[7]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[8]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[9]  Mark D. Plumbley,et al.  Theorems on Positive Data: On the Uniqueness of NMF , 2008, Comput. Intell. Neurosci..

[10]  John F. Canny,et al.  Some algebraic and geometric computations in PSPACE , 1988, STOC '88.

[11]  Suresh Venkatasubramanian.  Computational geometry column 55: new developments in nonnegative matrix factorization , 2013, SIGA.

[12]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[13]  Yaroslav Shitov.  Nonnegative rank depends on the field II , 2016 .

[14]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[15]  Bjorn Poonen,et al.  HILBERT'S TENTH PROBLEM OVER RINGS OF NUMBER-THEORETIC INTEREST , 2003 .

[16]  Nicolas Gillis,et al.  Sparse and unique nonnegative matrix factorization through data preprocessing , 2012, J. Mach. Learn. Res..

[17]  Marcel Paul Schützenberger,et al.  On the Definition of a Family of Automata , 1961, Inf. Control..

[18]  Sanjeev Arora,et al.  Computing a nonnegative matrix factorization -- provably , 2011, STOC '12.

[19]  Daniël Paulusma,et al.  Covering graphs with few complete bipartite subgraphs , 2007, Theor. Comput. Sci..

[20]  Yaroslav Shitov.  Nonnegative rank depends on the field , 2021, Math. Program..

[21]  Michael T. Goodrich,et al.  On the Complexity of Approximating and Illuminating Three-Dimensional Convex Polyhedra (Preliminary Version) , 1995, WADS.

[22]  Kousha Etessami,et al.  On the Complexity of Nash Equilibria and Other Fixed Points , 2010, SIAM J. Comput..

[23]  Kousha Etessami,et al.  Recursive Concurrent Stochastic Games , 2008, Log. Methods Comput. Sci..

[24]  Ankur Moitra.  An Almost Optimal Algorithm for Computing Nonnegative Rank , 2013, SODA.

[25]  Mihalis Yannakakis,et al.  Expressing combinatorial optimization problems by linear programs , 1991, STOC '88.

[26]  Stefan Kiefer,et al.  Trace Refinement in Labelled Markov Decision Processes , 2016, FoSSaCS.

[27]  David Barber,et al.  On the Computational Complexity of Stochastic Controller Optimization in POMDPs , 2011, TOCT.

[28]  Pierre Ailliot,et al.  Space–time modelling of precipitation by using a hidden Markov model and censored Gaussian distributions , 2009 .

[29]  Nicolas Gillis,et al.  The Why and How of Nonnegative Matrix Factorization , 2014, ArXiv.

[30]  A. Berman.  Rank Factorization of Nonnegative Matrices , 1973 .

[31]  Fabio Rapallo,et al.  Perturbation of Matrices and Nonnegative Rank with a View toward Statistical Models , 2010, SIAM J. Matrix Anal. Appl..

[32]  Jianxin Wu.  Hidden Markov model , 2018 .

[33]  Bernd Sturmfels,et al.  FIXED POINTS OF THE EM ALGORITHM AND NONNEGATIVE RANK BOUNDARIES , 2013, 1312.5634.

[34]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[35]  L. Shapley,et al.  Stochastic Games* , 1953, Proceedings of the National Academy of Sciences.

[36]  Nicolas Gillis,et al.  On the Geometric Interpretation of the Nonnegative Rank , 2010, 1009.0880.

[37]  James Worrell,et al.  Nonnegative Matrix Factorization Requires Irrationality , 2017, SIAM J. Appl. Algebra Geom..