SMOOTHING QUADRATIC REGULARIZATION METHODS FOR BOX CONSTRAINED NON-LIPSCHITZ OPTIMIZATION IN IMAGE RESTORATION

Abstract. We propose a smoothing quadratic regularization (SQR) method for solving box constrained optimization problems with a non-Lipschitz regularization term that includes the lp norm (0 < p < 1) of the gradient of the underlying image in the l2-lp problem as a special case. At each iteration of the SQR algorithm, a new iterate is generated by solving a strongly convex quadratic problem with box constraints. We prove that any cluster point of ε scaled first order stationary points with ε > 0 satisfies a first order necessary condition for a local minimizer as ε goes to 0, and the worst-case iteration complexity of the SQR algorithm for finding an ε scaled first order stationary point is O(ε−2). Numerical examples are given to show good performance of the SQR algorithm for image restoration.

[1]  Roger Fletcher,et al.  Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming , 2005, Numerische Mathematik.

[2]  Y. Ye,et al.  Lower Bound Theory of Nonzero Entries in Solutions of ℓ2-ℓp Minimization , 2010, SIAM J. Sci. Comput..

[3]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[4]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[5]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[6]  Xiaojun Chen,et al.  Smoothing Nonlinear Conjugate Gradient Method for Image Restoration Using Nonsmooth Nonconvex Minimization , 2010, SIAM J. Imaging Sci..

[7]  Mila Nikolova,et al.  Analysis of the Recovery of Edges in Images and Signals by Minimizing Nonconvex Regularized Least-Squares , 2005, Multiscale Model. Simul..

[8]  Cun-Hui Zhang,et al.  A group bridge approach for variable selection , 2009, Biometrika.

[9]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[10]  J. Horowitz,et al.  Asymptotic properties of bridge estimators in sparse high-dimensional regression models , 2008, 0804.0693.

[11]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[12]  Mila Nikolova,et al.  Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization , 2008, SIAM J. Imaging Sci..

[13]  Yinyu Ye,et al.  A note on the complexity of Lp minimization , 2011, Math. Program..

[14]  Karl Kunisch,et al.  A Bilevel Optimization Approach for Parameter Learning in Variational Models , 2013, SIAM J. Imaging Sci..

[15]  Xiaojun Chen,et al.  Worst-Case Complexity of Smoothing Quadratic Regularization Methods for Non-Lipschitzian Optimization , 2013, SIAM J. Optim..

[16]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[17]  Nicholas I. M. Gould,et al.  On the Evaluation Complexity of Composite Function Minimization with Applications to Nonconvex Nonlinear Programming , 2011, SIAM J. Optim..

[18]  Raymond H. Chan,et al.  A Multilevel Algorithm for Simultaneously Denoising and Deblurring Images , 2010, SIAM J. Sci. Comput..

[19]  Xiaojun Chen,et al.  Smoothing methods for nonsmooth, nonconvex minimization , 2012, Math. Program..

[20]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[21]  Michael Hintermüller,et al.  A superlinearly convergent R-regularized Newton scheme for variational models with concave sparsity-promoting priors , 2013, Computational Optimization and Applications.

[22]  Y. Ye,et al.  A Note on the Complexity of L p Minimization , 2009 .

[23]  Julie Zhou.  Robust Estimationに , 2009 .

[24]  Michael Hintermüller,et al.  Nonconvex TVq-Models in Image Restoration: Analysis and a Trust-Region Regularization-Based Superlinearly Convergent Solver , 2013, SIAM J. Imaging Sci..

[25]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[26]  Xiaojun Chen,et al.  Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization , 2015, Math. Program..

[27]  NG MICHAELK.,et al.  NONCONVEX `P -REGULARIZATION AND BOX CONSTRAINED MODEL FOR IMAGE RESTORATION , 2011 .

[28]  Xiaojun Chen,et al.  Smoothing Neural Network for Constrained Non-Lipschitz Optimization With Applications , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Zhaosong Lu,et al.  Iterative reweighted minimization methods for $$l_p$$lp regularized unconstrained nonlinear programming , 2012, Math. Program..

[30]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[31]  Mila Nikolova,et al.  Fast Nonconvex Nonsmooth Minimization Methods for Image Restoration and Reconstruction , 2010, IEEE Transactions on Image Processing.