Bregman Forward-Backward Operator Splitting

We propose an iterative method for finding a zero of the sum of two maximally monotone operators in reflexive Banach spaces. One of the operators is single-valued, and the method alternates an explicit step on this operator and an implicit step on the other one. Both steps involve the gradient of a convex function that is free to vary over the iterations. The convergence of the resulting forward-backward splitting method is analyzed using the theory of Legendre functions, under a novel assumption on the single-valued operator that captures various existing properties. When applied to minimization problems, rates are obtained for the objective values. The proposed framework unifies and extends several iterative methods which have thus far not been brought together, and it is also new in Euclidean spaces.

[1]  A. Goodman,et al.  On uniformly convex functions , 1991 .

[2]  Ernö Robert Csetnek,et al.  On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems , 2015 .

[3]  B. Mercier.  Lectures on Topics in Finite Element Solution of Elliptic Problems , 1980 .

[4]  Q. Nguyen.  Forward-Backward Splitting with Bregman Distances , 2015, 1505.05198.

[5]  R. Rockafellar.  Local boundedness of nonlinear, monotone operators. , 1969 .

[6]  Heinz H. Bauschke,et al.  Bregman Monotone Optimization Algorithms , 2003, SIAM J. Control. Optim..

[7]  J. Baillon,et al.  Quelques propriétés des opérateurs angle-bornés etn-cycliquement monotones , 1977 .

[8]  P. L. Combettes,et al.  Variable metric forward–backward splitting with applications to monotone inclusions in duality , 2012, 1206.6791.

[9]  David H. Gutman,et al.  Perturbed Fenchel duality and first-order methods , 2018, Mathematical Programming.

[10]  Marc Teboulle,et al.  A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications , 2017, Math. Oper. Res..

[11]  Heinz H. Bauschke,et al.  Applying FISTA to optimization problems (with or) without minimizers , 2018, Mathematical Programming.

[12]  Y. Censor,et al.  Parallel Optimization: Theory, Algorithms, and Applications , 1997 .

[13]  H. Attouch,et al.  Weak Versus Strong Convergence of a Regularized Newton Dynamic for Maximal Monotone Operators , 2018 .

[14]  F. Browder.  Nonlinear functional analysis and its applications , 1986 .

[15]  Quang Van Nguyen,et al.  Variable quasi-Bregman monotone sequences , 2015, Numerical Algorithms.

[16]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[17]  Heinz H. Bauschke,et al.  Legendre functions and the method of random Bregman projections , 1997 .

[18]  P. L. Combettes,et al.  Monotone Operator Methods for Nash Equilibria in Non-potential Games , 2011, 1106.0144.

[19]  Paul B. Garrett.  Hilbert spaces , 2022 .

[20]  R. Rockafellar.  On the maximality of sums of nonlinear monotone operators , 1970 .

[21]  Y. Censor,et al.  Parallel Optimization:theory , 1997 .

[22]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[23]  Lacra Pavel,et al.  An operator splitting approach for distributed generalized Nash equilibria computation , 2019, Autom..

[24]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[25]  Patrick L. Combettes,et al.  Solving Composite Monotone Inclusions in Reflexive Banach Spaces by Constructing Best Bregman Approximations from Their Kuhn-Tucker Set , 2015, 1505.00362.

[26]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[27]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[28]  Forward-Backward Splitting for Optimal Transport Based Problems , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  Julien Mairal,et al.  Proximal Methods for Hierarchical Sparse Coding , 2010, J. Mach. Learn. Res..

[30]  Patrick L. Combettes,et al.  A Parallel Splitting Method for Coupled Monotone Inclusions , 2009, SIAM J. Control. Optim..

[31]  David H. Gutman,et al.  A unified framework for Bregman proximal methods: subgradient, gradient, and accelerated gradient schemes , 2018 .

[32]  Massimiliano Pontil,et al.  Bilevel learning of the Group Lasso structure , 2018, NeurIPS.

[33]  Heinz H. Bauschke,et al.  Regularizing with Bregman-Moreau Envelopes , 2017, SIAM J. Optim..

[34]  P. L. Combettes,et al.  Variable metric quasi-Fejér monotonicity , 2012, 1206.5705.

[35]  Yurii Nesterov,et al.  Relatively Smooth Convex Optimization by First-Order Methods, and Applications , 2016, SIAM J. Optim..

[36]  B. Mercier.  Topics in Finite Element Solution of Elliptic Problems , 1979 .

[37]  Patrick L. Combettes,et al.  Monotone operator theory in convex optimization , 2018, Math. Program..

[38]  Guy Cohen,et al.  An Extension of the Auxiliary Problem Principle to Nonsymmetric Auxiliary Operators , 1997 .

[39]  H. Brezis,et al.  Image d’une somme d’operateurs monotones et applications , 1976 .

[40]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[41]  N. Bourbaki.  Espaces vectoriels topologiques , 1955 .

[42]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[43]  Loic Landrieu,et al.  Preconditioning of a Generalized Forward-Backward Splitting and Application to Optimization on Graphs , 2015, SIAM J. Imaging Sci..

[44]  N. Bourbaki,et al.  Topological Vector Spaces: Chapters 1–5 , 1987 .

[45]  Heinz H. Bauschke,et al.  ESSENTIAL SMOOTHNESS, ESSENTIAL STRICT CONVEXITY, AND LEGENDRE FUNCTIONS IN BANACH SPACES , 2001 .

[46]  Saverio Salzo,et al.  The Variable Metric Forward-Backward Splitting Algorithm Under Mild Differentiability Assumptions , 2016, SIAM J. Optim..

[47]  S. Banach,et al.  Théorie des opérations linéaires , 1932 .

[48]  N. Ghoussoub.  Self-dual Partial Differential Systems and Their Variational Principles , 2008 .