A Parallel Inertial Proximal Optimization Method

The Douglas-Rachford algorithm is a popular iterative method for finding a zero of a sum of two maximal monotone operators defined on a Hilbert space. In this paper, we propose an extension of this algorithm including inertia parameters and develop parallel versions to deal with the case of a sum of an arbitrary number of maximal operators. Based on this algorithm, parallel proximal algorithms are proposed to minimize over a linear subspace of a Hilbert space the sum of a finite number of proper, lower semicontinuous convex functions composed with linear operators. It is shown that particular cases of these methods are the simultaneous direction method of multipliers proposed by Stetzer et al., the parallel proximal algorithm developed by Combettes and Pesquet, and a parallelized version of an algorithm proposed by Attouch and Soueycatt.

[1]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[2]  F. Browder.  Convergence theorems for sequences of nonlinear operators in Banach spaces , 1967 .

[3]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[4]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[5]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[6]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[7]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[8]  J. Spingarn.  Partial inverse of a monotone operator , 1983 .

[9]  Jonathan E. Spingarn,et al.  Applications of the method of partial inverses to convex programming: Decomposition , 1985, Math. Program..

[10]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[11]  Shih-Ping Han.  A parallel algorithm for a class of convex programs , 1988 .

[12]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[13]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[14]  Jonathan Eckstein.  Parallel alternating direction multiplier decomposition of convex programs , 1994 .

[15]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Parallel Distributed Comput. Pract..

[16]  P. L. Combettes,et al.  Quasi-Fejérian Analysis of Some Optimization Algorithms , 2001 .

[17]  CONVERGENCE AND ASYMPTOTIC STABILIZATION FOR SOME DAMPED HYPERBOLIC EQUATIONS WITH NON-ISOLATED EQUILIBRIA , 2001 .

[18]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[19]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[20]  A. Moudafi,et al.  AN APPROXIMATE INERTIAL PROXIMAL METHOD USING THE ENLARGEMENT OF A MAXIMAL MONOTONE OPERATOR , 2003 .

[21]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[22]  A. Moudafi.  A hybrid inertial projection-proximal method for variational inequalities. , 2004 .

[23]  N. Deo.  Journal of Inequalities in Pure and Applied Mathematics , 2004 .

[24]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[25]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[26]  Caroline Chaux,et al.  Image analysis using a dual-tree M-band wavelet transform , 2006, IEEE Transactions on Image Processing.

[27]  Valérie R. Wajs,et al.  A variational formulation for frame-based inverse problems , 2007 .

[28]  Patrick L. Combettes,et al.  Proximal Thresholding Algorithm for Minimization over Orthonormal Bases , 2007, SIAM J. Optim..

[29]  J.-C. Pesquet,et al.  A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery , 2007, IEEE Journal of Selected Topics in Signal Processing.

[30]  P. Maingé.  Inertial Iterative Process for Fixed Points of Certain Quasi-nonexpansive Mappings , 2007 .

[31]  P. Maingé.  Regularized and Inertial algorithms for common fixed points of nonlinear operators , 2008 .

[32]  Benar Fux Svaiter,et al.  A family of projective splitting methods for the sum of two maximal monotone operators , 2007, Math. Program..

[33]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[34]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[35]  P. Maingé.  Convergence theorems for inertial KM-type algorithms , 2008 .

[36]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[37]  P. L. Combettes,et al.  Iterative construction of the resolvent of a sum of maximal monotone operators , 2009 .

[38]  Heinz H. Bauschke.  A Note on the Paper by Eckstein and Svaiter on "General Projective Splitting Methods for Sums of Maximal Monotone Operators" , 2009, SIAM J. Control. Optim..

[39]  Benar Fux Svaiter,et al.  General Projective Splitting Methods for Sums of Maximal Monotone Operators , 2009, SIAM J. Control. Optim..

[40]  Mário A. T. Figueiredo,et al.  Deconvolution of Poissonian images using variable splitting and augmented Lagrangian optimization , 2009, 2009 IEEE/SP 15th Workshop on Statistical Signal Processing.

[41]  Gabriele Steidl,et al.  Removing Multiplicative Noise by Douglas-Rachford Splitting Methods , 2010, Journal of Mathematical Imaging and Vision.

[42]  Luis,et al.  Convex Variational Formulation with Smooth Coupling for Multicomponent Signal Decomposition and Recovery , 2009 .

[43]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[44]  Heinz H. Bauschke,et al.  The Baillon-Haddad Theorem Revisited , 2009, 0906.0807.

[45]  Nelly Pustelnik,et al.  Nested Iterative Algorithms for Convex Constrained Image Recovery Problems , 2008, SIAM J. Imaging Sci..

[46]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[47]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[48]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[49]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[50]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[51]  Benar Fux Svaiter,et al.  On Weak Convergence of the Douglas-Rachford Method , 2010, SIAM J. Control. Optim..

[52]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[53]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[54]  Nelly Pustelnik,et al.  Parallel Proximal Algorithm for Image Restoration Using Hybrid Regularization , 2009, IEEE Transactions on Image Processing.

[55]  Heinz H. Bauschke,et al.  Fixed-Point Algorithms for Inverse Problems in Science and Engineering , 2011, Springer Optimization and Its Applications.