Convergence Analysis of Primal–Dual Based Methods for Total Variation Minimization with Finite Element Approximation

We consider a minimization model with total variational regularization, which can be reformulated as a saddle-point problem and then be efficiently solved by the primal–dual method. We utilize the consistent finite element method to discretize the saddle-point reformulation; thus possible jumps of the solution can be captured over some adaptive meshes and a generic domain can be easily treated. Our emphasis is analyzing the convergence of a more general primal–dual scheme with a combination factor for the discretized model. We establish the global convergence and derive the worst-case convergence rate measured by the iteration complexity for this general primal–dual scheme. This analysis is new in the finite element context for the minimization model with total variational regularization under discussion. Furthermore, we propose a prediction–correction scheme based on the general primal–dual scheme, which can significantly relax the step size for the discretization in the time direction. Its global convergence and the worst-case convergence rate are also established. Some preliminary numerical results are reported to verify the rationale of considering the general primal–dual scheme and the primal–dual-based prediction–correction scheme.

[1]  Bingsheng He,et al.  A Strictly Contractive Peaceman-Rachford Splitting Method for Convex Programming , 2014, SIAM J. Optim..

[2]  W. Ziemer Weakly differentiable functions , 1989 .

[3]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[4]  G. Golub,et al.  Inexact and preconditioned Uzawa algorithms for saddle point problems , 1994 .

[5]  L. R. Scott,et al.  The Mathematical Theory of Finite Element Methods , 1994 .

[6]  OsherStanley,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[7]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[8]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[9]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[10]  Ricardo H. Nochetto,et al.  Discrete Total Variation Flows without Regularization , 2012, SIAM J. Numer. Anal..

[11]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[12]  Xiaobing Feng,et al.  Analysis of total variation flow and its finite element approximations , 2003 .

[13]  C. M. Elliott,et al.  Numerical analysis of the TV regularization and H-1 fidelity model for decomposing an image into cartoon plus texture , 2009 .

[14]  G. Sapiro,et al.  Histogram Modification via Differential Equations , 1997 .

[15]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[16]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[17]  T. Chan,et al.  On the Convergence of the Lagged Diffusivity Fixed Point Method in Total Variation Image Restoration , 1999 .

[18]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[19]  T. Chan,et al.  Edge-preserving and scale-dependent properties of total variation regularization , 2003 .

[20]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[21]  S. Bartels Broken Sobolev space iteration for total variation regularized minimization problems , 2016 .

[22]  Weiwei Sun,et al.  Linearized FE Approximations to a Nonlinear Gradient Flow , 2013, SIAM J. Numer. Anal..

[23]  R. Glowinski,et al.  Numerical Methods for Nonlinear Variational Problems , 1985 .

[24]  V. Caselles,et al.  Minimizing total variation flow , 2000, Differential and Integral Equations.

[25]  Ernö Robert Csetnek,et al.  On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems , 2015 .

[26]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[27]  Sören Bartels,et al.  Total Variation Minimization with Finite Elements: Convergence and Iterative Solution , 2012, SIAM J. Numer. Anal..

[28]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[29]  W. Ring Structural Properties of Solutions to Total Variation Regularization Problems , 2000 .

[30]  D. Dobson,et al.  Convergence of an Iterative Method for Total Variation Denoising , 1997 .

[31]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[32]  Bingsheng He,et al.  Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective , 2012, SIAM J. Imaging Sci..

[33]  Curtis R. Vogel,et al.  Iterative Methods for Total Variation Denoising , 1996, SIAM J. Sci. Comput..

[34]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[35]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[36]  Raymond H. Chan,et al.  A Multilevel Algorithm for Simultaneously Denoising and Deblurring Images , 2010, SIAM J. Sci. Comput..

[37]  M. Novaga,et al.  The Total Variation Flow in RN , 2002 .

[38]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[39]  Apostol T. Vassilev,et al.  Analysis of the Inexact Uzawa Algorithm for Saddle Point Problems , 1997 .

[40]  J. Cahn,et al.  A microscopic theory for antiphase boundary motion and its application to antiphase domain coasening , 1979 .

[41]  Ke Chen,et al.  An Iterative Lagrange Multiplier Method for Constrained Total-Variation-Based Image Denoising , 2012, SIAM J. Numer. Anal..

[42]  Andreas Prohl,et al.  Rate of convergence of regularization procedures and finite element approximations for the total variation flow , 2005, Numerische Mathematik.

[43]  W. Queck The convergence factor of preconditioned algorithms of the Arrow-Hurwicz type , 1989 .

[44]  Mingqiang Zhu,et al.  An Efficient Primal-Dual Hybrid Gradient Algorithm For Total Variation Image Restoration , 2008 .

[45]  W. Ziemer Weakly Differentiable Functions: Sobolev Spaces and Functions of Bounded Variation , 1989 .

[46]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[47]  Jun Zou,et al.  Numerical identifications of parameters in parabolic systems , 1998 .

[48]  C. Vogel,et al.  Analysis of bounded variation penalty methods for ill-posed problems , 1994 .

[49]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[50]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[51]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[52]  D. Dobson,et al.  An image-enhancement technique for electrical impedance tomography , 1994 .

[53]  Michael K. Ng,et al.  On Semismooth Newton’s Methods for Total Variation Minimization , 2007, Journal of Mathematical Imaging and Vision.

[54]  Walter Zulehner,et al.  Analysis of iterative methods for saddle point problems: a unified approach , 2002, Math. Comput..

[55]  Stanley Osher,et al.  Explicit Algorithms for a New Time Dependent Model Based on Level Set Motion for Nonlinear Deblurring and Noise Removal , 2000, SIAM J. Sci. Comput..

[56]  K. Kunisch,et al.  Regularization of linear least squares problems by total bounded variation , 1997 .

[57]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[58]  G. Aubert,et al.  Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations (Applied Mathematical Sciences) , 2006 .

[59]  Xue-Cheng Tai,et al.  Identification of Discontinuous Coefficients in Elliptic Problems Using Total Variation Regularization , 2003, SIAM J. Sci. Comput..

[60]  Fadil Santosa,et al.  Recovery of Blocky Images from Noisy and Blurred Data , 1996, SIAM J. Appl. Math..

[61]  Giorgio C. Buttazzo,et al.  Variational Analysis in Sobolev and BV Spaces - Applications to PDEs and Optimization, Second Edition , 2014, MPS-SIAM series on optimization.

[62]  Zhiming Chen,et al.  An Augmented Lagrangian Method for Identifying Discontinuous Parameters in Elliptic Systems , 1999 .

[63]  D. Dobson,et al.  Analysis of regularized total variation penalty methods for denoising , 1996 .