Linearized primal-dual methods for linear inverse problems with total variation regularization and finite element discretization

Linear inverse problems with total variation regularization can be reformulated as saddle-point problems; the primal and dual variables of such a saddle-point reformulation can be discretized in piecewise affine and constant finite element spaces, respectively. Thus, the well-developed primal-dual approach (a.k.a. the inexact Uzawa method) is conceptually applicable to such a regularized and discretized model. When the primal-dual approach is applied, the resulting subproblems may be highly nontrivial and it is necessary to discuss how to tackle them and thus make the primal-dual approach implementable. In this paper, we suggest linearizing the data-fidelity quadratic term of the hard subproblems so as to obtain easier ones. A linearized primal-dual method is thus proposed. Inspired by the fact that the linearized primal-dual method can be explained as an application of the proximal point algorithm, a relaxed version of the linearized primal-dual method, which can often accelerate the convergence numerically with the same order of computation, is also proposed. The global convergence and worst-case convergence rate measured by the iteration complexity are established for the new algorithms. Their efficiency is verified by some numerical results.

[1]  Xiaoming Yuan,et al.  Convergence Analysis of Primal–Dual Based Methods for Total Variation Minimization with Finite Element Approximation , 2018, J. Sci. Comput..

[2]  Kristian Bredies,et al.  Preconditioned Douglas-Rachford Splitting Methods for Convex-concave Saddle-point Problems , 2015, SIAM J. Numer. Anal..

[3]  Giorgio C. Buttazzo,et al.  Variational Analysis in Sobolev and BV Spaces - Applications to PDEs and Optimization, Second Edition , 2014, MPS-SIAM series on optimization.

[4]  Bingsheng He,et al.  Customized proximal point algorithms for linearly constrained convex minimization and saddle-point problems : a uniform approach , 2011 .

[5]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[6]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[7]  Sören Bartels,et al.  Total Variation Minimization with Finite Elements: Convergence and Iterative Solution , 2012, SIAM J. Numer. Anal..

[8]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[9]  Bingsheng He,et al.  Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective , 2012, SIAM J. Imaging Sci..

[10]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[11]  Bradley J. Lucier,et al.  Error Bounds for Finite-Difference Methods for Rudin-Osher-Fatemi Image Smoothing , 2011, SIAM J. Numer. Anal..

[12]  Henryk Gzyl,et al.  Linear Inverse Problems - The Maximum Entropy Connection , 2011, Series on Advances in Mathematics for Applied Sciences.

[13]  K. Kunisch,et al.  A duality-based approach to elliptic control problems in non-reflexive Banach spaces , 2011 .

[14]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[15]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[16]  F. Tröltzsch.  Optimal Control of Partial Differential Equations: Theory, Methods and Applications , 2010 .

[17]  Bangti Jin,et al.  A duality-based splitting method for ` 1-TV image restoration with automatic regularization parameter choice ∗ , 2009 .

[18]  P. Hansen.  Discrete Inverse Problems: Insight and Algorithms , 2010 .

[19]  Mingqiang Zhu,et al.  An Efficient Primal-Dual Hybrid Gradient Algorithm For Total Variation Image Restoration , 2008 .

[20]  Antonin Chambolle,et al.  The Discontinuity Set of Solutions of the TV Denoising Problem and Some Extensions , 2007, Multiscale Model. Simul..

[21]  William K. Allard,et al.  Total Variation Regularization for Image Denoising, I. Geometric Theory , 2007, SIAM J. Math. Anal..

[22]  Tony F. Chan,et al.  Aspects of Total Variation Regularized L[sup 1] Function Approximation , 2005, SIAM J. Appl. Math..

[23]  Karl Kunisch,et al.  Differentiability properties of the L1-tracking functional and application to the Robin inverse problem , 2004 .

[24]  T. Chan,et al.  Aspects of Total Variation Regularized L 1 Function Approximation 1 , 2004 .

[25]  T. Chan,et al.  Edge-preserving and scale-dependent properties of total variation regularization , 2003 .

[26]  Xue-Cheng Tai,et al.  Identification of Discontinuous Coefficients in Elliptic Problems Using Total Variation Regularization , 2003, SIAM J. Sci. Comput..

[27]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[28]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[29]  W. Ring.  Structural Properties of Solutions to Total Variation Regularization Problems , 2000 .

[30]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[31]  K. Kunisch,et al.  Regularization by Functions of Bounded Variation and Applications to Image Enhancement , 1999 .

[32]  Zhiming Chen,et al.  An Augmented Lagrangian Method for Identifying Discontinuous Parameters in Elliptic Systems , 1999 .

[33]  Jun Zou,et al.  Numerical identifications of parameters in parabolic systems , 1998 .

[34]  Jun Zou,et al.  Numerical identifications of parameters in parabolic systems , 1998 .

[35]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[36]  K. Kunisch,et al.  Regularization of linear least squares problems by total bounded variation , 1997 .

[37]  Fadil Santosa,et al.  Recovery of Blocky Images from Noisy and Blurred Data , 1996, SIAM J. Appl. Math..

[38]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[39]  A. Tikhonov,et al.  Numerical Methods for the Solution of Ill-Posed Problems , 1995 .

[40]  G. Golub,et al.  Inexact and preconditioned Uzawa algorithms for saddle point problems , 1994 .

[41]  C. Vogel,et al.  Analysis of bounded variation penalty methods for ill-posed problems , 1994 .

[42]  D. Dobson,et al.  An image-enhancement technique for electrical impedance tomography , 1994 .

[43]  D C Dobson,et al.  An image-enhancement technique for electrical impedance tomography , 1994 .

[44]  L. R. Scott,et al.  The Mathematical Theory of Finite Element Methods , 1994 .

[45]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[46]  S. Gutman.  Identification of discontinuous parameters in flow equations , 1990 .

[47]  W. Ziemer.  Weakly Differentiable Functions: Sobolev Spaces and Functions of Bounded Variation , 1989 .

[48]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[49]  W. Ziemer.  Weakly differentiable functions , 1989 .

[50]  C. Vogel.  Computational Methods for Inverse Problems , 1987 .

[51]  J. Baumeister.  Stable solution of inverse problems , 1987 .

[52]  G. Wing,et al.  A Primer on Integral Equations of the First Kind: The Problem of Deconvolution and Unfolding , 1987 .

[53]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[54]  C. W. Groetsch,et al.  The theory of Tikhonov regularization for Fredholm equations of the first kind , 1984 .

[55]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[56]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[57]  E. G. Gol'shtein,et al.  Modified Lagrangians in Convex Programming and their Generalizations , 1979 .

[58]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[59]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[60]  J. Lions.  Optimal Control of Systems Governed by Partial Differential Equations , 1971 .

[61]  B. Martinet,et al.  R'egularisation d''in'equations variationnelles par approximations successives , 1970 .

[62]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[63]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[64]  R. Kanwal.  Linear Integral Equations , 1925, Nature.