Complexity of Unconstrained L2-Lp Minimization

We consider the unconstrained L2-Lp minimization: find a minimizer of ‖Ax−b‖2+λ‖x‖p for given A ∈ Rm×n, b ∈ R and parameters λ > 0, p ∈ [0, 1). This problem has been studied extensively in variable selection and sparse least squares fitting for high dimensional data. Theoretical results show that the minimizers of the L2-Lp problem have various attractive features due to the concavity and non-Lipschitzian property of the regularization function ‖ · ‖p. In this paper, we show that the Lq-Lp minimization problem is strongly NP-hard for any p ∈ [0, 1) and q ≥ 1, including its smoothed version. On the other hand, we show that, by choosing parameters (p, λ) carefully, a minimizer, global or local, will have certain desired sparsity. We believe that these results provide new theoretical insights to the studies and applications of the concave regularized optimization problems.

[1]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[2]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[3]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[4]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[5]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[6]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[7]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[8]  M. Lai,et al.  An Unconstrained $\ell_q$ Minimization with $0q\leq1$ for Sparse Solution of Underdetermined Linear Systems , 2011 .

[9]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[10]  David S. Johnson,et al.  `` Strong '' NP-Completeness Results: Motivation, Examples, and Implications , 1978, JACM.

[11]  J. Horowitz,et al.  Asymptotic properties of bridge estimators in sparse high-dimensional regression models , 2008, 0804.0693.

[12]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[13]  Y. Ye,et al.  Lower Bound Theory of Nonzero Entries in Solutions of ℓ2-ℓp Minimization , 2010, SIAM J. Sci. Comput..

[14]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[15]  Yinyu Ye,et al.  A note on the complexity of Lp minimization , 2011, Math. Program..

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .