Distributed Projection on the Simplex and $\ell _1$ Ball via ADMM and Gossip

We derive distributed algorithms for projecting the local values of the agents of a computing network on the simplex or on the <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> ball. These algorithms are based on the distributed alternating direction method of multipliers to solve a convex optimization problem of the form <inline-formula><tex-math notation="LaTeX">$\min _x \sum _n f_n(x)$</tex-math></inline-formula>, where each function <inline-formula><tex-math notation="LaTeX">$f_n$</tex-math></inline-formula> is local to node <inline-formula> <tex-math notation="LaTeX">$n$</tex-math></inline-formula> and has an easy-to-compute proximity operator.

[1]  Franck Iutzeler,et al.  Distributed Computation of Quantiles via ADMM , 2017, IEEE Signal Processing Letters.

[2]  Walid Hachem,et al.  Analysis of Sum-Weight-Like Algorithms for Averaging in Wireless Sensor Networks , 2012, IEEE Transactions on Signal Processing.

[3]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[4]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[5]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[6]  Pascal Bianchi,et al.  Performance of a Distributed Stochastic Approximation Algorithm , 2012, IEEE Transactions on Information Theory.

[7]  Alexandros G. Dimakis,et al.  Geographic Gossip: Efficient Averaging for Sensor Networks , 2007, IEEE Transactions on Signal Processing.

[8]  Asuman E. Ozdaglar,et al.  Distributed Alternating Direction Method of Multipliers , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[9]  Philippe Ciblat,et al.  Analysis of Max-Consensus Algorithms in Wireless Channels , 2012, IEEE Transactions on Signal Processing.

[10]  Pascal Bianchi,et al.  A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization , 2014, IEEE Transactions on Automatic Control.

[11]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[12]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[13]  M. Degroot.  Reaching a Consensus , 1974 .

[14]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[15]  Michael G. Rabbat,et al.  Broadcast Gossip Algorithms for Consensus on Strongly Connected Digraphs , 2012, IEEE Transactions on Signal Processing.

[16]  Asuman E. Ozdaglar,et al.  On the O(1=k) convergence of asynchronous distributed alternating Direction Method of Multipliers , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[17]  Scott Shenker,et al.  Epidemic algorithms for replicated database maintenance , 1988, OPSR.

[18]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[19]  Alejandro Ribeiro,et al.  Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals , 2008, IEEE Transactions on Signal Processing.

[20]  Anand D. Sarwate,et al.  Broadcast Gossip Algorithms for Consensus , 2009, IEEE Transactions on Signal Processing.

[21]  Pascal Bianchi,et al.  Convergence of a Multi-Agent Projected Stochastic Gradient Algorithm for Non-Convex Optimization , 2011, IEEE Transactions on Automatic Control.

[22]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[23]  Patrick L. Combettes,et al.  Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping , 2014 .

[24]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..

[25]  George Giakkoupis,et al.  Rumor spreading and vertex expansion , 2012, SODA.

[26]  Laurent Condat.  Fast projection onto the simplex and the l1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pmb {l}_\mathbf {1}$$\end{ , 2015, Mathematical Programming.

[27]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[28]  Jorge Cortés,et al.  Distributed algorithms for reaching consensus on general functions , 2008, Autom..

[29]  Jocelyn Chanussot,et al.  Robust linear unmixing with enhanced sparsity , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[30]  Laurent Condat,et al.  A Convex Approach to K-Means Clustering and Image Segmentation , 2017, EMMCVPR.

[31]  Pascal Bianchi,et al.  Asynchronous distributed optimization using a randomized alternating direction method of multipliers , 2013, 52nd IEEE Conference on Decision and Control.

[32]  Pascal Bianchi,et al.  Explicit Convergence Rate of a Distributed Alternating Direction Method of Multipliers , 2013, IEEE Transactions on Automatic Control.