Smoothing Nonlinear Conjugate Gradient Method for Image Restoration Using Nonsmooth Nonconvex Minimization

Image restoration problems are often converted into large-scale, nonsmooth, and nonconvex optimization problems. Most existing minimization methods are not efficient for solving such problems. It is well known that nonlinear conjugate gradient methods are preferred to solve large-scale smooth optimization problems due to their simplicity, low storage, practical computation efficiency, and nice convergence properties. In this paper, we propose a smoothing nonlinear conjugate gradient method where an intelligent scheme is used to update the smoothing parameter at each iteration and guarantees that any accumulation point of a sequence generated by this method is a Clarke stationary point of the nonsmooth and nonconvex optimization problem. Moreover, we present a class of smoothing functions and show their approximation properties. This method is easy to implement without adding any new variables. Three image restoration problems with different pixels and different regularization terms are used in numerical tests. Experimental results and comparison with the continuation method in [M. Nikolova et al., SIAM J. Imaging Sci., 1 (2008), pp. 2-25] show the efficiency of the proposed method.

[1]  Xiaojun Chen,et al.  Smoothing Projected Gradient Method and Its Application to Stochastic Linear Complementarity Problems , 2009, SIAM J. Optim..

[2]  Krzysztof C. Kiwiel,et al.  Convergence of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization , 2007, SIAM J. Optim..

[3]  Olvi L. Mangasarian,et al.  A class of smoothing functions for nonlinear and mixed complementarity problems , 1996, Comput. Optim. Appl..

[4]  Aggelos K. Katsaggelos,et al.  Image sequence filtering in quantum-limited noise with applications to low-dose fluoroscopy , 1993, IEEE Trans. Medical Imaging.

[5]  L. Evans.  Measure theory and fine properties of functions , 1992 .

[6]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[7]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[8]  C. H. Slump.  Real-time image restoration in diagnostic X-ray imaging, the effects on quantum noise , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[9]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[10]  A. Guitton,et al.  Adaptive subtraction of multiples using the L1‐norm , 2004 .

[11]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[12]  Y. Ye,et al.  On Homotopy-Smoothing Methods for Box-Constrained Variational Inequalities , 1999 .

[13]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[14]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[15]  Li Zhang,et al.  Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search , 2006, Numerische Mathematik.

[16]  Raymond H. Chan,et al.  A Fast Algorithm for Deblurring Models with Neumann Boundary Conditions , 1999, SIAM J. Sci. Comput..

[17]  M. Overton.  NONSMOOTH OPTIMIZATION VIA BFGS , 2008 .

[18]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[19]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[20]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[21]  Tony F. Chan,et al.  Aspects of Total Variation Regularized L[sup 1] Function Approximation , 2005, SIAM J. Appl. Math..

[22]  M. Fukushima,et al.  A modified BFGS method and its global convergence in nonconvex minimization , 2001 .

[23]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[24]  Xiaojun Chen.  SMOOTHING METHODS FOR COMPLEMENTARITY PROBLEMS AND THEIR APPLICATIONS : A SURVEY , 2000 .

[25]  Mila Nikolova,et al.  Minimizers of Cost-Functions Involving Nonsmooth Data-Fidelity Terms. Application to the Processing of Outliers , 2002, SIAM J. Numer. Anal..

[26]  L. Qi,et al.  A Globally Convergent Successive Approximation Method for Severely Nonsmooth Equations , 1995 .

[27]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[28]  Xiaojun Chen,et al.  Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities , 1998, Math. Comput..

[29]  Mila Nikolova,et al.  Efficient Minimization Methods of Mixed l2-l1 and l1-l1 Norms for Image Restoration , 2005, SIAM J. Sci. Comput..

[30]  Mila Nikolova,et al.  Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization , 2008, SIAM J. Imaging Sci..

[31]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[32]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[33]  Weijun Zhou,et al.  A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence , 2006 .