Distributed soft thresholding for sparse signal recovery

In this paper, we address the problem of distributed sparse recovery of signals acquired via compressed measurements in a sensor network. We propose a new class of distributed algorithms to solve Lasso regression problems, when the communication to a fusion center is not possible, e.g., due to communication cost or privacy reasons. More precisely, we introduce a distributed iterative soft thresholding algorithm (DISTA) that consists of three steps: an averaging step, a gradient step, and a soft thresholding operation. We prove the convergence of DISTA in networks represented by regular graphs, and we compare it with existing methods in terms of performance, memory, and complexity.

[1]  Yonina C. Eldar,et al.  Distributed Compressed Sensing for Static and Time-Varying Networks , 2013, IEEE Transactions on Signal Processing.

[2]  Georgios B. Giannakis,et al.  Consensus-Based Distributed Support Vector Machines , 2010, J. Mach. Learn. Res..

[3]  Z. Opial.  Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[4]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[5]  Mikael Skoglund,et al.  A greedy pursuit algorithm for distributed compressed sensing , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[7]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[8]  Gonzalo Mateos,et al.  Distributed Sparse Linear Regression , 2010, IEEE Transactions on Signal Processing.

[9]  João M. F. Xavier,et al.  Distributed Basis Pursuit , 2010, IEEE Transactions on Signal Processing.

[10]  Francesco Bullo,et al.  Distributed Control of Robotic Networks , 2009 .

[11]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[12]  Angelia Nedic,et al.  Distributed subgradient projection algorithm for convex optimization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Enrico Magli,et al.  Distributed Iterative Thresholding for $\ell _{0}/\ell _{1}$ -Regularized Linear Inverse Problems , 2015, IEEE Transactions on Information Theory.

[14]  Massimo Fornasier,et al.  Theoretical Foundations and Numerical Methods for Sparse Recovery , 2010, Radon Series on Computational and Applied Mathematics.

[15]  Benjamin Van Roy,et al.  Consensus Propagation , 2005, IEEE Transactions on Information Theory.

[16]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[17]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[18]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[19]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[20]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[21]  G.B. Giannakis,et al.  Distributed compression-estimation using wireless sensor networks , 2006, IEEE Signal Processing Magazine.

[22]  Richard G. Baraniuk,et al.  Distributed Compressive Sensing , 2009, ArXiv.

[23]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[24]  Georgios B. Giannakis,et al.  Consensus-based distributed linear support vector machines , 2010, IPSN '10.

[25]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..