Dykstra’s Splitting and an Approximate Proximal Point Algorithm for Minimizing the Sum of Convex Functions

We show that Dykstra’s splitting for projecting onto the intersection of convex sets can be extended to minimize the sum of convex functions and a regularizing quadratic function. We give conditions for which convergence to the primal minimizer holds so that more than one convex function can be minimized at a time, the convex functions are not necessarily sampled in a cyclic manner, and the SHQP strategy for problems involving the intersection of more than one convex set can be applied. When the sum does not involve the regularizing quadratic function, we discuss an approximate proximal point method combined with Dykstra’s splitting to minimize this sum.

[1]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[2]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[3]  Ran Davidi,et al.  Perturbation resilience and superiorization of iterative algorithms , 2010, Inverse problems.

[4]  Alvaro R. De Pierro,et al.  Incremental Subgradients for Constrained Convex Optimization: A Unified Framework and New Methods , 2009, SIAM J. Optim..

[5]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[6]  P. L. Combettes,et al.  Proximity for sums of composite functions , 2010, 1007.3535.

[7]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[8]  Heinz H. Bauschke,et al.  Strong conical hull intersection property, bounded linear regularity, Jameson’s property (G), and error bounds in convex optimization , 1999, Math. Program..

[9]  P. L. Combettes,et al.  A Dykstra-like algorithm for two monotone operators , 2007 .

[10]  Ran Davidi,et al.  Projected Subgradient Minimization Versus Superiorization , 2013, Journal of Optimization Theory and Applications.

[11]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[12]  A. Kruger.  About Regularity of Collections of Sets , 2006 .

[13]  Zhi-Quan Luo,et al.  Iteration complexity analysis of block coordinate descent methods , 2013, Mathematical Programming.

[14]  Chin How Jeffrey Pang,et al.  The Supporting Halfspace-Quadratic Programming Strategy for the Dual of the Best Approximation Problem , 2016, SIAM J. Optim..

[15]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[16]  Chin How Jeffrey Pang,et al.  Distributed Deterministic Asynchronous Algorithms in Time-Varying Graphs Through Dykstra Splitting , 2018, SIAM J. Optim..

[17]  Wei Hong Yang,et al.  Regularities and their relations to error bounds , 2004, Math. Program..

[18]  Sien Deng,et al.  Weak sharp minima revisited, part II: application to linear regularity and error bounds , 2005, Math. Program..

[19]  Angelia Nedic,et al.  Random algorithms for convex minimization problems , 2011, Math. Program..

[20]  P. L. Combettes,et al.  Dualization of Signal Recovery Problems , 2009, 0907.0436.

[21]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[22]  Paul Tseng,et al.  A coordinate gradient descent method for nonsmooth separable minimization , 2008, Math. Program..

[23]  Alvaro R. De Pierro,et al.  On the convergence of Han's method for convex programming with quadratic objective , 1991, Math. Program..

[24]  Paul Tseng,et al.  Dual coordinate ascent methods for non-strictly convex minimization , 1993, Math. Program..

[25]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[26]  P. Tseng,et al.  Block-Coordinate Gradient Descent Method for Linearly Constrained Nonsmooth Separable Optimization , 2009 .

[27]  Guy Pierra,et al.  Decomposition through formalization in a product space , 1984, Math. Program..

[28]  Shih-Ping Han,et al.  A Decomposition Method and Its Application to Convex Programming , 1989, Math. Oper. Res..

[29]  Émilie Chouzenoux,et al.  Dual Block-Coordinate Forward-Backward Algorithm with Application to Deconvolution and Deinterlacing of Video Sequences , 2017, Journal of Mathematical Imaging and Vision.

[30]  R. Dykstra,et al.  A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces , 1986 .

[31]  Frank Deutsch,et al.  Two generalizations of Dykstra’s cyclic projections algorithm , 1997, Math. Program..

[32]  Angelia Nedic,et al.  Incremental Stochastic Subgradient Algorithms for Convex Optimization , 2008, SIAM J. Optim..

[33]  Shih-Ping Han,et al.  A successive projection method , 1988, Math. Program..

[34]  M. Raydan,et al.  Alternating Projection Methods , 2011 .

[35]  Stephen J. Wright.  Coordinate descent algorithms , 2015, Mathematical Programming.

[36]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[37]  Patrick L. Combettes,et al.  On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints , 2009, Computational Optimization and Applications.

[38]  Angelia Nedic,et al.  Convergence Rate of Distributed Averaging Dynamics and Optimization in Networks , 2015, Found. Trends Syst. Control..

[39]  Amir Beck,et al.  On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes , 2015, SIAM J. Optim..

[40]  R. Mathar,et al.  A cyclic projection algorithm via duality , 1989 .

[41]  F. Deutsch.  Accelerating the Convergence of the Method of Alternating Projections Via a Line Search: a Brief Survey , 2001 .

[42]  B. Martinet.  Brève communication. Régularisation d'inéquations variationnelles par approximations successives , 1970 .