Subdifferentiable functions and partial data communication in a distributed deterministic asynchronous Dykstra's algorithm

We described a decentralized distributed deterministic asynchronous Dykstra's algorithm that allows for time-varying graphs in an earlier paper. In this paper, we show how to incorporate subdifferentiable functions into the framework using a step similar to the bundle method. We point out that our algorithm also allows for partial data communications. We discuss a standard step for treating the composition of a convex and linear function.

[1]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[2]  Shih-Ping Han,et al.  A successive projection method , 1988, Math. Program..

[3]  Angelia Nedic,et al.  Convergence Rate of Distributed Averaging Dynamics and Optimization in Networks , 2015, Found. Trends Syst. Control..

[4]  Amir Beck,et al.  On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes , 2015, SIAM J. Optim..

[5]  Ming Yan,et al.  ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates , 2015, SIAM J. Sci. Comput..

[6]  Wei Shi,et al.  Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs , 2016, SIAM J. Optim..

[7]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[8]  Vivek S. Borkar,et al.  A Distributed Boyle-Dykstra-Han Scheme , 2017, SIAM J. Optim..

[9]  M. Raydan,et al.  Alternating Projection Methods , 2011 .

[10]  Frank Deutsch,et al.  Two generalizations of Dykstra’s cyclic projections algorithm , 1997, Math. Program..

[11]  Karin Schwab,et al.  Best Approximation In Inner Product Spaces , 2016 .

[12]  Angelia Nedic,et al.  Distributed Optimization Over Time-Varying Directed Graphs , 2015, IEEE Trans. Autom. Control..

[13]  Pascal Bianchi,et al.  Asynchronous distributed optimization using a randomized alternating direction method of multipliers , 2013, 52nd IEEE Conference on Decision and Control.

[14]  Chin How Jeffrey Pang,et al.  The Supporting Halfspace-Quadratic Programming Strategy for the Dual of the Best Approximation Problem , 2016, SIAM J. Optim..

[15]  R. Mathar,et al.  A cyclic projection algorithm via duality , 1989 .

[16]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[17]  Soummya Kar,et al.  Gossip Algorithms for Distributed Signal Processing , 2010, Proceedings of the IEEE.

[18]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[19]  Giuseppe Notarstefano,et al.  Asynchronous Distributed Optimization Via Randomized Dual Proximal Gradient , 2015, IEEE Transactions on Automatic Control.

[20]  Patrick L. Combettes,et al.  Asynchronous block-iterative primal-dual decomposition methods for monotone inclusions , 2015, Mathematical Programming.

[21]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[22]  Qing Ling,et al.  EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization , 2014, 1404.6264.

[23]  IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2014, Reims, France, September 21-24, 2014 , 2014, MLSP.

[24]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[25]  R. Dykstra.  An Algorithm for Restricted Least Squares Regression , 1983 .

[26]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[27]  Hamid Reza Feyzmahdavian,et al.  Analysis and Implementation of an Asynchronous Optimization Algorithm for the Parameter Server , 2016, ArXiv.