Matrix Sparsification and the Sparse Null Space Problem

We revisit the matrix problems sparse null space and matrix sparsification, and show that they are equivalent. We then proceed to seek algorithms for these problems: We prove the hardness of approximation of these problems, and also give a powerful tool to extend algorithms and heuristics for sparse approximation theory to these problems.

[1]  Xin Zhang,et al.  Incremental methods for simple problems in time series: algorithms and experiments , 2005, 9th International Database Engineering & Application Symposium (IDEAS'05).

[2]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[3]  Sudipto Guha,et al.  Near-optimal sparse fourier representations via sampling , 2002, STOC '02.

[4]  J. Gilbert.  Computing a sparse basis for the null space , 1987 .

[5]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[6]  Shmuel Friedland,et al.  The Sparse Basis Problem and Multilinear Algebra , 1995, SIAM J. Matrix Anal. Appl..

[7]  D. Shasha,et al.  Sparse solutions for linear prediction problems , 2006 .

[8]  Edoardo Amaldi,et al.  The Complexity and Approximability of Finding Maximum Feasible Subsystems of Linear Relations , 1995, Theor. Comput. Sci..

[9]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[10]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[11]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[12]  A. Gilbert,et al.  Theoretical and experimental analysis of a randomized algorithm for Sparse Fourier transform analysis , 2004, math/0411102.

[13]  Bishnu S. Atal,et al.  Amplitude optimization and pitch prediction in multipulse coders , 1989, IEEE Trans. Acoust. Speech Signal Process..

[14]  Rick Chartrand,et al.  Nonconvex Compressed Sensing and Error Correction , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[15]  Robert W. Heath,et al.  Designing structured tight frames via an alternating projection method , 2005, IEEE Transactions on Information Theory.

[16]  A. Pothen.  Sparse null bases and marriage theorems , 1984 .

[17]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[18]  Joel A. Tropp,et al.  Recovery of short, complex linear combinations via /spl lscr//sub 1/ minimization , 2005, IEEE Transactions on Information Theory.

[19]  Anna C. Gilbert,et al.  Improved time bounds for near-optimal sparse Fourier representations , 2005, SPIE Optics + Photonics.

[20]  Viggo Kann,et al.  Polynomially Bounded Minimization Problems That Are Hard to Approximate , 1993, Nord. J. Comput..

[21]  Iain S. Duff,et al.  Direct methods for sparse matrices27100 , 1986 .

[22]  Jacques Stern,et al.  The hardness of approximate optima in lattices, codes, and systems of linear equations , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  S. T. McCormick,et al.  A combinatorial approach to some sparse matrix problems , 1983 .

[25]  Guy Kindler,et al.  Approximating CVP to Within Almost-Polynomial Factors is NP-Hard , 1998, Electron. Colloquium Comput. Complex..

[26]  J. Tropp.  JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION , 2004 .

[27]  Xiaoshen Wang,et al.  A Simple Proof of Descartes's Rule of Signs , 2004, Am. Math. Mon..

[28]  Kurt Mehlhorn,et al.  A Faster Algorithm for Minimum Cycle Basis of Graphs , 2004, ICALP.

[29]  M. Heath,et al.  An algorithm to compute a sparse basis of the null space , 1985 .

[30]  E. Schmidt.  Zur Theorie der linearen und nichtlinearen Integralgleichungen , 1907 .

[31]  Bernd Gärtner,et al.  Understanding and using linear programming , 2007, Universitext.

[32]  John Wright,et al.  Dense Error Correction Via $\ell^1$-Minimization , 2010, IEEE Transactions on Information Theory.

[33]  Marek Karpinski,et al.  Approximating minimum unsatisfiability of linear equations , 2002, SODA '02.

[34]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[35]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[36]  A. Hoffman,et al.  A Fast Algorithm that makes Matrices Optimally Sparse , 1984 .

[37]  S. Thomas McCormick,et al.  A hierarchical algorithm for making sparse matrices sparser , 1992, Math. Program..

[38]  T. Coleman,et al.  The null space problem I. complexity , 1986 .

[39]  Vladimir N. Temlyakov,et al.  Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[40]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[41]  Franco P. Preparata,et al.  The Densest Hemisphere Problem , 1978, Theor. Comput. Sci..

[42]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[43]  F. MacWilliams,et al.  The Theory of Error-Correcting Codes , 1977 .

[44]  Bernhard Schölkopf,et al.  Sparse Greedy Matrix Approximation for Machine Learning , 2000, International Conference on Machine Learning.