A linearized bregman algorithm for decentralized basis pursuit

In this paper we solve a decentralized basis pursuit problem in a multiagent system where each agent holds part of the linear observations on a common sparse vector. The agents collaborate to recover the sparse vector through limited neighboring communication. The proposed decentralized linearized Bregman algorithm solves the Lagrange dual of an augmented ℓ1 model that is equivalent to basis pursuit. The fact that this dual problem is unconstrained and differentiable enables a lightweight yet efficient decentralized gradient algorithm. We prove nearly linear convergence of the dual and primal variables to their optima. Numerical experiments demonstrate the effectiveness of the proposed algorithm.

[1]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[2]  Alejandro Ribeiro,et al.  Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals , 2008, IEEE Transactions on Signal Processing.

[3]  Georgios B. Giannakis,et al.  Distributed Spectrum Sensing for Cognitive Radio Networks by Exploiting Sparsity , 2010, IEEE Transactions on Signal Processing.

[4]  Paul Tseng,et al.  Exact Regularization of Convex Programs , 2007, SIAM J. Optim..

[5]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[6]  Wotao Yin,et al.  Analysis and Generalizations of the Linearized Bregman Method , 2010, SIAM J. Imaging Sci..

[7]  Qing Ling,et al.  Linearly convergent decentralized consensus optimization with the alternating direction method of multipliers , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  Stephen P. Boyd,et al.  Fastest Mixing Markov Chain on a Graph , 2004, SIAM Rev..

[9]  H. Vincent Poor,et al.  A Collaborative Training Algorithm for Distributed Learning , 2009, IEEE Transactions on Information Theory.

[10]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[11]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[12]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[13]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[14]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[15]  Zhi Tian,et al.  Compressed Wideband Sensing in Cooperative Cognitive Radio Networks , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[16]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[17]  Gonzalo Mateos,et al.  Distributed Sparse Linear Regression , 2010, IEEE Transactions on Signal Processing.

[18]  João M. F. Xavier,et al.  Distributed Basis Pursuit , 2010, IEEE Transactions on Signal Processing.

[19]  Wotao Yin,et al.  Augmented 퓁1 and Nuclear-Norm Models with a Globally Linearly Convergent Algorithm , 2012, SIAM J. Imaging Sci..