Optimality Conditions and Complexity for Non-Lipschitz Constrained Optimization Problems

In this paper, we consider a class of nonsmooth, nonconvex constrained optimization problems where the objective function may be not Lipschitz continuous and the feasible set is a general closed convex set. Using the theory of the generalized directional derivative and the Clarke tangent cone, we derive a first order necessary optimality condition for local minimizers of the problem, and define the generalized stationary point of it. The generalized stationary point is the Clarke stationary point when the objective function is Lipschitz continuous at this point, and the scaled stationary point in the existing literature when the objective function is not Lipschitz continuous at this point. We prove the consistency between the generalized directional derivative and the limit of the classic directional derivatives associated with the smoothing function. Moreover we present the computational complexity and lower bound theory of the problem.

[1]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[2]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[3]  P. Atzberger.  Introduction to Nonlinear Optimization , 2020, Linear Algebra and Optimization with Applications to Machine Learning.

[4]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[5]  Y. Ye,et al.  Lower Bound Theory of Nonzero Entries in Solutions of ℓ2-ℓp Minimization , 2010, SIAM J. Sci. Comput..

[6]  Zizhuo Wang,et al.  Complexity of Unconstrained L2-Lp Minimization , 2011 .

[7]  James V. Burke,et al.  Epi-convergent Smoothing with Applications to Convex Composite Functions , 2012, SIAM J. Optim..

[8]  Xiaojun Chen,et al.  Complexity of unconstrained $$L_2-L_p$$ minimization , 2011, Math. Program..

[9]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[10]  Xiaojun Chen,et al.  Smoothing methods for nonsmooth, nonconvex minimization , 2012, Math. Program..

[11]  J. Aubin,et al.  Differential inclusions set-valued maps and viability theory , 1984 .

[12]  J. Horowitz,et al.  Asymptotic properties of bridge estimators in sparse high-dimensional regression models , 2008, 0804.0693.

[13]  Yinyu Ye,et al.  A note on the complexity of Lp minimization , 2011, Math. Program..

[14]  Raymond H. Chan,et al.  A Multilevel Algorithm for Simultaneously Denoising and Deblurring Images , 2010, SIAM J. Sci. Comput..

[15]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  Jianqing Fan,et al.  Nonconcave penalized likelihood with a diverging number of parameters , 2004, math/0406466.

[17]  Xiaojun Chen,et al.  Worst-Case Complexity of Smoothing Quadratic Regularization Methods for Non-Lipschitzian Optimization , 2013, SIAM J. Optim..

[18]  Xiaojun Chen,et al.  Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization , 2015, Math. Program..

[19]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[20]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[21]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[22]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[23]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[24]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[25]  Jianqing Fan,et al.  COMMENTS ON « WAVELETS IN STATISTICS : A REVIEW , 2009 .

[26]  Julie Zhou.  Robust Estimationに , 2009 .

[27]  Alfred Auslender.  How to deal with the unbounded in optimization: Theory and algorithms , 1997, Math. Program..

[28]  Mila Nikolova,et al.  Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization , 2008, SIAM J. Imaging Sci..

[29]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[30]  Adrian S. Lewis,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[31]  Cun-Hui Zhang,et al.  A group bridge approach for variable selection , 2009, Biometrika.

[32]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[33]  NG MICHAELK.,et al.  NONCONVEX `P -REGULARIZATION AND BOX CONSTRAINED MODEL FOR IMAGE RESTORATION , 2011 .

[34]  J. Burke,et al.  Gradient Consistency for Integral-convolution Smoothing Functions , 2013 .

[35]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[36]  Michael L. Overton,et al.  A Sequential Quadratic Programming Algorithm for Nonconvex, Nonsmooth Constrained Optimization , 2012, SIAM J. Optim..