A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms

We propose a new first-order splitting algorithm for solving jointly the primal and dual formulations of large-scale convex minimization problems involving the sum of a smooth function with Lipschitzian gradient, a nonsmooth proximable function, and linear composite functions. This is a full splitting approach, in the sense that the gradient and the linear operators involved are applied explicitly without any inversion, while the nonsmooth functions are processed individually via their proximity operators. This work brings together and notably extends several classical splitting schemes, like the forward–backward and Douglas–Rachford methods, as well as the recent primal–dual method of Chambolle and Pock designed for problems with linear composite terms.

[1]  E. Sidky,et al.  Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle–Pock algorithm , 2011, Physics in medicine and biology.

[2]  Bingsheng He,et al.  Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective , 2012, SIAM J. Imaging Sci..

[3]  Mohamed-Jalal Fadili,et al.  A Proximal Iteration for Deconvolving Poisson Noisy Images Using Sparse Representations , 2008, IEEE Transactions on Image Processing.

[4]  Jonathan Eckstein Splitting methods for monotone operators with applications to parallel optimization , 1989 .

[5]  B. Mercier Topics in Finite Element Solution of Elliptic Problems , 1979 .

[6]  Teemu Pennanen,et al.  Dualization of Generalized Equations of Maximal Monotone Type , 1999, SIAM J. Optim..

[7]  P. Tseng Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[8]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[9]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[10]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[11]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[12]  Benar Fux Svaiter,et al.  On Weak Convergence of the Douglas-Rachford Method , 2010, SIAM J. Control. Optim..

[13]  Mohamed-Jalal Fadili,et al.  A Generalized Forward-Backward Splitting , 2011, SIAM J. Imaging Sci..

[14]  D. Gabay Applications of the method of multipliers to variational inequalities , 1983 .

[15]  Lynn McLinden An extension of Fenchel’s duality theorem to saddle functions and dual minimax problems , 1974 .

[16]  I. Yamada,et al.  NON-STRICTLY CONVEX MINIMIZATION OVER THE FIXED POINT SET OF AN ASYMPTOTICALLY SHRINKING NONEXPANSIVE MAPPING , 2002 .

[17]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[18]  Massimo Fornasier,et al.  Theoretical Foundations and Numerical Methods for Sparse Recovery , 2010, Radon Series on Computational and Applied Mathematics.

[19]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[20]  P. L. Combettes,et al.  Proximity for sums of composite functions , 2010, 1007.3535.

[21]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[22]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[23]  R. Rockafellar Conjugate Duality and Optimization , 1987 .

[24]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[25]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[26]  Laurent Condat A generic first-order primal-dual method for convex optimization involving Lipschitzian, proximable and linear composite terms , 2011 .

[27]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[28]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[29]  Heinz H. Bauschke,et al.  Fixed-Point Algorithms for Inverse Problems in Science and Engineering , 2011, Springer Optimization and Its Applications.

[30]  Nelly Pustelnik,et al.  Nested Iterative Algorithms for Convex Constrained Image Recovery Problems , 2008, SIAM J. Imaging Sci..

[31]  G. Chen Forward-backward splitting techniques: theory and applications , 1994 .

[32]  Mohamed-Jalal Fadili,et al.  Total Variation Projection With First Order Schemes , 2011, IEEE Transactions on Image Processing.

[33]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[34]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[35]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[36]  J. Moreau Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[37]  A. Chambolle,et al.  An introduction to Total Variation for Image Analysis , 2009 .

[38]  R. Monteiro,et al.  Iteration-complexity of block-decomposition algorithms and the alternating minimization augmented Lagrangian method , 2010 .

[39]  Stephen M. Robinson,et al.  Composition duality and maximal monotonicity , 1999, Math. Program..

[40]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[41]  Gregory B. Passty Ergodic convergence to a zero of the sum of monotone operators in Hilbert space , 1979 .

[42]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[43]  R. Rockafellar Minimax Theorems and Conjugate Saddle-Functions. , 1964 .