Nonnegative Factorization and The Maximum Edge Biclique Problem

Nonnegative matrix factorization (NMF) is a data analysis technique based on the approximation of a nonnegative matrix with a product of two nonnegative factors, which allows compression and interpretation of nonnegative data. In this paper, we study the case of rank-one factorization and show that when the matrix to be factored is not required to be nonnegative, the corresponding problem (R1NF) becomes NP-hard. This sheds new light on the complexity of NMF since any algorithm for fixed-rank NMF must be able to solve at least implicitly such rank-one subproblems. Our proof relies on a reduction of the maximum edge biclique problem to R1NF. We also link stationary points of R1NF to feasible solutions of the biclique problem, which allows us to design a new type of biclique finding algorithm based on the application of a block-coordinate descent scheme to R1NF. We show that this algorithm, whose algorithmic complexity per iteration is proportional to the number of edges in the graph, is guaranteed to converge to a biclique and that it performs competitively with existing methods on random graphs and text mining datasets.

[1]  Yurii Nesterov,et al.  Solving Infinite-dimensional Optimization Problems by Polynomial Approximation , 2010 .

[2]  S. Sra.  Nonnegative Matrix Approximation: Algorithms and Applications , 2006 .

[3]  Pauli Miettinen,et al.  The Boolean column and column-row matrix decompositions , 2008, Data Mining and Knowledge Discovery.

[4]  G. Oggioni,et al.  Generalized Nash Equilibrium and market coupling in the European power system , 2010 .

[5]  Gregor Zöttl,et al.  A Framework of Peak Load Pricing with Strategic Firms , 2008, Oper. Res..

[6]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[7]  Chris H. Q. Ding,et al.  Binary Matrix Factorization with Applications , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[8]  Peter L. Hammer,et al.  Consensus algorithms for the generation of all maximal bicliques , 2004, Discret. Appl. Math..

[9]  A. Berman.  Rank Factorization of Nonnegative Matrices , 1973 .

[10]  Gregory Ponthiere,et al.  On the Golden Rule of capital accumulation under endogenous longevity , 2010, Math. Soc. Sci..

[11]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[12]  Vincent D. Blondel,et al.  Polynomial-Time Computation of the Joint Spectral Radius for Some Sets of Nonnegative Matrices , 2009, SIAM J. Matrix Anal. Appl..

[13]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[14]  J. Tharakan,et al.  On the Impact of Labor Market Matching on Regional Disparities , 2009 .

[15]  Inderjit S. Dhillon,et al.  Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem , 2007, SDM.

[16]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[17]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[18]  Haesun Park,et al.  Sparse Nonnegative Matrix Factorization for Clustering , 2008 .

[19]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.

[20]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[21]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[22]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[23]  Leonidas C. Koutsougeras,et al.  Decentralization of the core through Nash equilibrium , 2008 .

[24]  Yves Smeers,et al.  Stochastic Equilibrium Models for Generation Capacity Expansion , 2010 .

[25]  V. N. Bogaevski,et al.  Matrix Perturbation Theory , 1991 .

[26]  Nicolas Gillis,et al.  Using underapproximations for sparse nonnegative matrix factorization , 2009, Pattern Recognit..

[27]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[28]  Pierre Pestieau,et al.  The economics of wealth transfer tax , 2010 .

[29]  Marie-Louise Leroux,et al.  Endogenous differential mortality, non monitored effort and optimal non linear taxation , 2008 .

[30]  Chris H. Q. Ding,et al.  Biclustering Protein Complex Interactions with a Biclique Finding Algorithm , 2006, Sixth International Conference on Data Mining (ICDM'06).

[31]  Chris H. Q. Ding,et al.  Binary matrix factorization for analyzing gene expression data , 2009, Data Mining and Knowledge Discovery.

[32]  Yassine Lefouili,et al.  Leniency programs for multimarket firms: The effect of Amnesty Plus on cartel formation ☆ , 2012 .

[33]  J. Rombouts,et al.  Asymptotic properties of the Bernstein density copula for dependent data , 2008 .

[34]  Pierre Pestieau,et al.  Long term care insurance puzzle , 2010 .

[35]  Joydeep Ghosh,et al.  Under Consideration for Publication in Knowledge and Information Systems Generative Model-based Document Clustering: a Comparative Study , 2003 .

[36]  Ali Ghodsi,et al.  Nonnegative matrix factorization via rank-one downdate , 2008, ICML '08.

[37]  R. Boucekkine,et al.  How do epidemics induce behavioral changes? , 2009 .

[38]  Andrzej Cichocki,et al.  Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization , 2007, ICA.

[39]  Rüdiger Stephan,et al.  An extension of disjunctive programming and its impact for compact tree formulations , 2010, 1007.1136.

[40]  Thierry Bréchet,et al.  Technological greening, eco-efficiency and no-regret strategy , 2010 .

[41]  J. Resende,et al.  Does the absence of competition in the market foster competition for the market? A dynamic approach to aftermarkets , 2008 .

[42]  Elena Del Rey,et al.  On Welfare Criteria and Optimality in an Endogenous Growth Model , 2010 .

[43]  T. Baudin.  The optimal trade-off between quality and quantity with uncertain child survival , 2010 .

[44]  Jean-Pierre Florens,et al.  Nonparametric frontier estimation from noisy data , 2010 .

[45]  G. Ponthiere,et al.  Optimal tax policy and expected longevity: a mean and variance approach , 2008 .

[46]  G. Oggioni,et al.  Average power contracts can mitigate carbon leakage , 2008 .

[47]  Karthik Devarajan,et al.  Nonnegative Matrix Factorization: An Analytical and Interpretive Tool in Computational Biology , 2008, PLoS Comput. Biol..

[48]  P. Pestieau,et al.  Should We Subsidize Longevity? , 2009, SSRN Electronic Journal.

[49]  Lars Stentoft,et al.  Multivariate Option Pricing with Time Varying Volatility and Correlations , 2010 .

[50]  Donghui Chen,et al.  Nonnegativity constraints in numerical analysis , 2009, The Birth of Numerical Analysis.

[51]  Axel Pierru,et al.  Uncertain long-run emissions targets, CO2 price and global energy transition: a general equilibrium approach , 2010 .

[52]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[53]  Jinyan Li,et al.  Efficient Mining of Large Maximal Bicliques , 2006, DaWaK.

[54]  C. Hsiao,et al.  An easy test for two stationary long processes being uncorrelated via AR approximations , 2008 .

[55]  P. Pestieau,et al.  Fertility, Human Capital Accumulation, and the Pension System , 2009, SSRN Electronic Journal.

[56]  Luigi Grippo,et al.  On the convergence of the block nonlinear Gauss-Seidel method under convex constraints , 2000, Oper. Res. Lett..

[57]  P. Jean-Jacques Herings,et al.  Coalition Formation Among Farsighted Agents , 2010, Games.

[58]  Gregor Zoettl.  On investment decisions in liberalized electricity markets: the impact of price caps at the spot market , 2008 .

[59]  Joel E. Cohen,et al.  Nonnegative ranks, decompositions, and factorizations of nonnegative matrices , 1993 .

[60]  Y. Nesterov,et al.  Double smoothing technique for infinite-dimensional optimization problems with applications to optimal control , 2010 .

[61]  Paul Van Dooren,et al.  Descent methods for Nonnegative Matrix Factorization , 2008, ArXiv.

[62]  Paul Van Dooren,et al.  Weighted Nonnegative Matrix Factorization and Face Feature Extraction , 2007 .

[63]  G. Oggioni,et al.  Market coupling and the organization of counter-trading: separating energy and transmission again? , 2010 .

[64]  Henry Tulkens,et al.  The impact of the unilateral EU commitment on the stability of international climate agreements , 2010 .

[65]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[66]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[67]  René Peeters,et al.  The maximum edge biclique problem is NP-complete , 2003, Discret. Appl. Math..

[68]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[69]  Chris H. Q. Ding,et al.  Nonnegative Matrix Factorization for Combinatorial Optimization: Spectral Clustering, Graph Matching, and Clique Finding , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[70]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .