Efficient iterative solution of finite element discretized nonsmooth minimization problems

Abstract For the iterative solution of finite element discretized, nonsmooth minimization problems the alternating direction method of multipliers (ADMM) is considered, which is a flexible method to solve a large class of convex minimization problems. Particular features are its unconditional convergence with respect to the involved step size and its direct applicability. In particular, this article deals with the ADMM with variable step sizes and devises an adjustment rule for the step size relying on the monotonicity of the residual and discusses proper stopping criteria. The proposed scheme is applied to finite element formulations of the obstacle problem and the Rudin–Osher–Fatemi image denoising problem, and the numerical experiments show significant improvements over established variants of the ADMM.

[1]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[2]  R. Glowinski,et al.  Numerical Methods for Nonlinear Variational Problems , 1985 .

[3]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[4]  Richard G. Baraniuk,et al.  Fast Alternating Direction Optimization Methods , 2014, SIAM J. Imaging Sci..

[5]  J. Hiriart-Urruty,et al.  Fundamentals of Convex Analysis , 2004 .

[6]  Yuan Shen,et al.  On the O(1/t) convergence rate of Ye-Yuan's modified alternating direction method of multipliers , 2014, Appl. Math. Comput..

[7]  Haim Brezis,et al.  Sur la régularité de la solution d'inéquations elliptiques , 1968 .

[8]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[9]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[10]  B. He,et al.  Alternating Direction Method with Self-Adaptive Penalty Parameters for Monotone Variational Inequalities , 2000 .

[11]  Kazufumi Ito,et al.  The Primal-Dual Active Set Strategy as a Semismooth Newton Method , 2002, SIAM J. Optim..

[12]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[13]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[14]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[15]  Shiqian Ma,et al.  Fast alternating linearization methods for minimizing the sum of two convex functions , 2009, Math. Program..

[16]  Sören Bartels,et al.  Numerical Methods for Nonlinear Partial Differential Equations , 2015 .

[17]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[18]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[19]  Bingsheng He,et al.  Self-adaptive operator splitting methods for monotone variational inequalities , 2003, Numerische Mathematik.

[20]  Bingsheng He,et al.  On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers , 2014, Numerische Mathematik.

[21]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[22]  Jonathan Eckstein.  Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[23]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[24]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[25]  Xiaoming Yuan,et al.  A sequential updating scheme of the Lagrange multiplier for separable convex programming , 2016, Math. Comput..

[26]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[27]  Bingsheng He,et al.  Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities , 1998, Oper. Res. Lett..

[28]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[29]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[30]  Xiaoming Yuan,et al.  An augmented Lagrangian based parallel splitting method for separable convex minimization with applications to image processing , 2014, Math. Comput..

[31]  Ricardo H. Nochetto,et al.  Discrete Total Variation Flows without Regularization , 2012, SIAM J. Numer. Anal..

[32]  M. Hestenes.  Multiplier and gradient methods , 1969 .