Linearly Constrained Non-Lipschitz Optimization for Image Restoration

Nonsmooth nonconvex optimization models have been widely used in the restoration and reconstruction of real images. In this paper, we consider a linearly constrained optimization problem with a non-Lipschitz regularization term in the objective function which includes the $l_p$ norm ($0<p<1$) of the gradient of the underlying image in the $l_2$-$l_p$ problem as a special case. We prove that any cluster point of $\epsilon$ scaled first order stationary points satisfies a first order necessary condition for a local minimizer of the optimization problem as $\epsilon $ goes to $0$. We propose a smoothing quadratic regularization (SQR) method for solving the problem. At each iteration of the SQR algorithm, a new iterate is generated by solving a strongly convex quadratic problem with linear constraints. Moreover, we show that the SQR algorithm can find an $\epsilon$ scaled first order stationary point in at most $O(\epsilon^{-2})$ iterations from any starting point. Numerical examples are given to show good pe...

[1]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[2]  Xiaojun Chen,et al.  Worst-Case Complexity of Smoothing Quadratic Regularization Methods for Non-Lipschitzian Optimization , 2013, SIAM J. Optim..

[3]  Hugues Talbot,et al.  A Majorize-Minimize Subspace Approach for ℓ2-ℓ0 Image Regularization , 2011, SIAM J. Imaging Sci..

[4]  Xiaojun Chen,et al.  Optimality Conditions and a Smoothing Trust Region Newton Method for NonLipschitz Optimization , 2013, SIAM J. Optim..

[5]  Raymond H. Chan,et al.  Constrained Total Variation Deblurring Models and Fast Algorithms Based on Alternating Direction Method of Multipliers , 2013, SIAM J. Imaging Sci..

[6]  Mila Nikolova,et al.  Analysis of the Recovery of Edges in Images and Signals by Minimizing Nonconvex Regularized Least-Squares , 2005, Multiscale Model. Simul..

[7]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[8]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[9]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[10]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[11]  Xiaojun Chen,et al.  Smoothing Neural Network for Constrained Non-Lipschitz Optimization With Applications , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Michael Hintermüller,et al.  A Smoothing Descent Method for Nonconvex TV $$^q$$ -Models , 2011, Efficient Algorithms for Global Optimization Methods in Computer Vision.

[13]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[14]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[15]  J. Horowitz,et al.  Asymptotic properties of bridge estimators in sparse high-dimensional regression models , 2008, 0804.0693.

[16]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[17]  Karl Kunisch,et al.  A Bilevel Optimization Approach for Parameter Learning in Variational Models , 2013, SIAM J. Imaging Sci..

[18]  Mila Nikolova,et al.  Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization , 2008, SIAM J. Imaging Sci..

[19]  Yinyu Ye,et al.  A note on the complexity of Lp minimization , 2011, Math. Program..

[20]  Nicholas I. M. Gould,et al.  On the Evaluation Complexity of Composite Function Minimization with Applications to Nonconvex Nonlinear Programming , 2011, SIAM J. Optim..

[21]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[22]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[23]  Xiaojun Chen,et al.  Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization , 2015, Math. Program..

[24]  Xiaojun Chen,et al.  Smoothing methods for nonsmooth, nonconvex minimization , 2012, Math. Program..

[25]  Andy M. Yip,et al.  A Primal-Dual Active-Set Method for Non-Negativity Constrained Total Variation Deblurring Problems , 2007, IEEE Transactions on Image Processing.

[26]  Mila Nikolova,et al.  Fast Nonconvex Nonsmooth Minimization Methods for Image Restoration and Reconstruction , 2010, IEEE Transactions on Image Processing.

[27]  Cun-Hui Zhang,et al.  A group bridge approach for variable selection , 2009, Biometrika.

[28]  Michael Hintermüller,et al.  A superlinearly convergent R-regularized Newton scheme for variational models with concave sparsity-promoting priors , 2013, Computational Optimization and Applications.

[29]  Zhaosong Lu,et al.  Iterative reweighted minimization methods for $$l_p$$lp regularized unconstrained nonlinear programming , 2012, Math. Program..

[30]  Xiaojun Chen,et al.  Non-Lipschitz $\ell_{p}$-Regularization and Box Constrained Model for Image Restoration , 2012, IEEE Transactions on Image Processing.

[31]  Y. Ye,et al.  Lower Bound Theory of Nonzero Entries in Solutions of ℓ2-ℓp Minimization , 2010, SIAM J. Sci. Comput..

[32]  M. Lai,et al.  An Unconstrained $\ell_q$ Minimization with $0q\leq1$ for Sparse Solution of Underdetermined Linear Systems , 2011 .

[33]  Michael Hintermüller,et al.  Nonconvex TVq-Models in Image Restoration: Analysis and a Trust-Region Regularization-Based Superlinearly Convergent Solver , 2013, SIAM J. Imaging Sci..

[34]  Michael Ulbrich,et al.  A mesh-independence result for semismooth Newton methods , 2004, Math. Program..

[35]  Xiaojun Chen,et al.  Smoothing Nonlinear Conjugate Gradient Method for Image Restoration Using Nonsmooth Nonconvex Minimization , 2010, SIAM J. Imaging Sci..

[36]  Marco Prato,et al.  A New Semiblind Deconvolution Approach for Fourier-Based Image Restoration: An Application in Astronomy , 2013, SIAM J. Imaging Sci..

[37]  Raymond H. Chan,et al.  A Multilevel Algorithm for Simultaneously Denoising and Deblurring Images , 2010, SIAM J. Sci. Comput..