Asymmetric forward–backward–adjoint splitting for solving monotone inclusions involving three operators

In this work we propose a new splitting technique, namely Asymmetric Forward–Backward–Adjoint splitting, for solving monotone inclusions involving three terms, a maximally monotone, a cocoercive and a bounded linear operator. Our scheme can not be recovered from existing operator splitting methods, while classical methods like Douglas–Rachford and Forward–Backward splitting are special cases of the new algorithm. Asymmetric preconditioning is the main feature of Asymmetric Forward–Backward–Adjoint splitting, that allows us to unify, extend and shed light on the connections between many seemingly unrelated primal-dual algorithms for solving structured convex optimization problems proposed in recent years. One important special case leads to a Douglas–Rachford type scheme that includes a third cocoercive operator.

[1]  Xiaoming Yuan,et al.  A Note on the Alternating Direction Method of Multipliers , 2012, J. Optim. Theory Appl..

[2]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[3]  L. Briceño-Arias Forward-Douglas–Rachford splitting and forward-partial inverse method for solving monotone inclusions , 2012, 1212.5942.

[4]  Damek Davis,et al.  Convergence Rate Analysis of Primal-Dual Splitting Schemes , 2014, SIAM J. Optim..

[5]  Damek Davis,et al.  A Three-Operator Splitting Scheme and its Optimization Applications , 2015, 1504.01032.

[6]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[7]  J. Baillon,et al.  Quelques propriétés des opérateurs angle-bornés etn-cycliquement monotones , 1977 .

[8]  Bang Công Vu,et al.  A splitting algorithm for dual monotone inclusions involving cocoercive operators , 2011, Advances in Computational Mathematics.

[9]  P. Tseng Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[10]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[11]  P. L. Combettes,et al.  Compositions and convex combinations of averaged nonexpansive operators , 2014, 1407.5100.

[12]  Laurent Condat,et al.  A Generic Proximal Algorithm for Convex Optimization—Application to Total Variation Minimization , 2014, IEEE Signal Processing Letters.

[13]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[14]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[15]  Patrick L. Combettes,et al.  Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping , 2014 .

[16]  R. Rockafellar,et al.  Implicit Functions and Solution Mappings , 2009 .

[17]  Yunda Dong An LS-free splitting method for composite mappings , 2005, Appl. Math. Lett..

[18]  Kim-Chuan Toh,et al.  A Convergent 3-Block Semi-Proximal ADMM for Convex Minimization Problems with One Strongly Convex Block , 2014, Asia Pac. J. Oper. Res..

[19]  Heinz H. Bauschke,et al.  The Baillon-Haddad Theorem Revisited , 2009, 0906.0807.

[20]  P. Tseng,et al.  Modified Projection-Type Methods for Monotone Variational Inequalities , 1996 .

[21]  P. L. Combettes,et al.  Variable metric forward–backward splitting with applications to monotone inclusions in duality , 2012, 1206.6791.

[22]  Marc Teboulle,et al.  Supplementary Material to : A Simple Algorithm for a Class of Nonsmooth Convex-Concave Saddle-Point Problems , 2015 .

[23]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[24]  Shiqian Ma,et al.  Global Convergence of Unmodified 3-Block ADMM for a Class of Convex Minimization Problems , 2015, Journal of Scientific Computing.

[25]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[26]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[27]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[28]  Pascal Bianchi,et al.  A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization , 2014, IEEE Transactions on Automatic Control.

[29]  D. Klatte Book review: Implicit Functions and Solution Mappings:A View from Variational Analysis. Second Edition. By A. L. Dontchev and R. T. Rockafellar. Springer, New York, 2014 , 2015 .

[30]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[31]  Ernö Robert Csetnek,et al.  Recent Developments on Primal–Dual Splitting Methods with Applications to Convex Minimization , 2014 .

[32]  Shiqian Ma,et al.  Iteration Complexity Analysis of Multi-block ADMM for a Family of Convex Minimization Without Strong Convexity , 2015, Journal of Scientific Computing.

[33]  Asen L. Dontchev,et al.  Regularity and Conditioning of Solution Mappings in Variational Analysis , 2004 .

[34]  Caihua Chen,et al.  On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks , 2013 .

[35]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[36]  Xiaoming Yuan,et al.  The direct extension of ADMM for three-block separable convex minimization models is convergent when one function is strongly convex , 2014 .

[37]  J. Moreau Proximité et dualité dans un espace hilbertien , 1965 .

[38]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[39]  Damek Davis,et al.  Convergence Rate Analysis of Several Splitting Schemes , 2014, 1406.4834.

[40]  A. Cegielski Iterative Methods for Fixed Point Problems in Hilbert Spaces , 2012 .

[41]  J. Pesquet,et al.  A Class of Randomized Primal-Dual Algorithms for Distributed Optimization , 2014, 1406.6404.

[42]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[43]  Mohamed-Jalal Fadili,et al.  Convergence rates with inexact non-expansive operators , 2014, Mathematical Programming.

[44]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[45]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[46]  Marc Teboulle,et al.  A simple algorithm for a class of nonsmooth convex-concave saddle-point problems , 2015, Oper. Res. Lett..

[47]  Bingsheng He,et al.  Convergence Analysis of Primal-Dual Algorithms for a Saddle-Point Problem: From Contraction Perspective , 2012, SIAM J. Imaging Sci..