Low-Rank Matrix Approximation with Weights or Missing Data Is NP-Hard

Weighted low-rank approximation (WLRA), a dimensionality reduction technique for data analysis, has been successfully used in several applications, such as in collaborative filtering to design recommender systems or in computer vision to recover structure from motion. In this paper, we prove that computing an optimal WLRA is NP-hard, already when a rank-one approximation is sought. In fact, we show that it is hard to compute approximate solutions to the WLRA problem with some prescribed accuracy. Our proofs are based on reductions from the maximum-edge biclique problem and apply to strictly positive weights as well as to binary weights (the latter corresponding to low-rank matrix approximation with missing data).

[1]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, SIAM Rev..

[2]  Thierry Bréchet,et al.  Adaptation and Mitigation in Long-term Climate Policy , 2010, Environmental and Resource Economics.

[3]  Rüdiger Stephan,et al.  An extension of disjunctive programming and its impact for compact tree formulations , 2010, 1007.1136.

[4]  Gang Chen,et al.  Collaborative Filtering Using Orthogonal Nonnegative Matrix Tri-factorization , 2007 .

[5]  R. Manne,et al.  Missing values in principal component analysis , 1998 .

[6]  Thierry Bréchet,et al.  Tradable pollution permits in dynamic general equilibrium: can optimality and acceptability be reconciled? , 2013 .

[7]  Harry Shum,et al.  Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Pei Chen,et al.  Optimization Algorithms on Subspaces: Revisiting Missing Data Problem in Low-Rank Matrix , 2008, International Journal of Computer Vision.

[9]  Paul S. Wang,et al.  Weighted Low-Rank Approximation of General Complex Matrices and Its Application in the Design of 2-D Digital Filters , 1997 .

[10]  Nicolas Gillis,et al.  Nonnegative Factorization and The Maximum Edge Biclique Problem , 2008, 0810.4225.

[11]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[12]  Dima Grigoriev,et al.  Complexity of Quantifier Elimination in the Theory of Algebraically Closed Fields , 1984, MFCS.

[13]  Nicolas Gillis,et al.  Using underapproximations for sparse nonnegative matrix factorization , 2009, Pattern Recognit..

[14]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[15]  Jinsong Tan,et al.  Inapproximability of Maximum Weighted Edge Biclique and Its Applications , 2007, TAMC.

[16]  Daniel Bienstock,et al.  Potential Function Methods for Approximately Solving Linear Programming Problems: Theory and Practice , 2002 .

[17]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[18]  J. Huriot,et al.  Economics of Cities , 2000 .

[19]  T. Baudin.  The optimal trade-off between quality and quantity with uncertain child survival , 2010 .

[20]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[21]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[22]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[23]  Faculteit Ingenieurswetenschappen,et al.  WEIGHTED LOW RANK APPROXIMATION: ALGORITHMS AND APPLICATIONS , 2006 .

[24]  Thierry Bréchet,et al.  The Benefits of Cooperation Under Uncertainty: the Case of Climate Change , 2011, Environmental Modeling & Assessment.

[25]  René Peeters,et al.  The maximum edge biclique problem is NP-complete , 2003, Discret. Appl. Math..

[26]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[27]  J. Gabszewicz.  La différenciation des produits , 2006 .

[28]  Andrea Silvestrini,et al.  Aggregation of exponential smoothing processes with an application to portfolio risk evaluation , 2013 .

[29]  J. Johannes,et al.  Iterative regularization in nonparametric instrumental regression , 2013 .

[30]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[31]  F. Schroyen,et al.  Optimal pricing and capacity choice for a public service under risk of interruption , 2011 .

[32]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[33]  GillisNicolas,et al.  Low-Rank Matrix Approximation with Weights or Missing Data Is NP-Hard , 2011 .

[34]  Gene H. Golub,et al.  Matrix computations , 1983 .

[35]  Nicolas Gillis,et al.  A multilevel approach for nonnegative matrix factorization , 2010, J. Comput. Appl. Math..

[36]  David W. Jacobs,et al.  Linear Fitting with Missing Data for Structure-from-Motion , 2001, Comput. Vis. Image Underst..

[37]  Christian Gourieroux,et al.  Simulation-based econometric methods , 1996 .

[38]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[39]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[40]  Pierre Pestieau,et al.  The impact of a minimum pension on old age poverty and its budgetary cost. Evidence from Latin America , 2011 .

[41]  Paul Belleflamme,et al.  Industrial Organization: Markets and Strategies , 2010 .

[42]  Mahesan Niranjan,et al.  Approximate low-rank factorization with structured factors , 2010, Comput. Stat. Data Anal..

[43]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[44]  S. Zamir,et al.  Lower Rank Approximation of Matrices by Least Squares With Any Choice of Weights , 1979 .

[45]  Jacques-François Thisse,et al.  Economic Geography: The Integration of Regions and Nations , 2008 .

[46]  Laurence A. Wolsey,et al.  Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, 4th International Conference, CPAIOR 2007, Brussels, Belgium, May 23-26, 2007, Proceedings , 2007, CPAIOR.

[47]  Fredrik Kahl,et al.  Structure from Motion with Missing Data is NP-Hard , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[48]  James Renegar,et al.  A mathematical view of interior-point methods in convex optimization , 2001, MPS-SIAM series on optimization.

[49]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..