Distributed Recovery of Jointly Sparse Signals Under Communication Constraints

The problem of the distributed recovery of jointly sparse signals has attracted much attention recently. Let us assume that the nodes of a network observe different sparse signals with common support; starting from linear, compressed measurements, and exploiting network communication, each node aims at reconstructing the support and the non-zero values of its observed signal. In the literature, distributed greedy algorithms have been proposed to tackle this problem, among which the most reliable ones require a large amount of transmitted data, which barely adapts to realistic network communication constraints. In this paper, we address the problem through a reweighted ℓ1 soft thresholding technique, in which the threshold is iteratively tuned based on the current estimate of the support. The proposed method adapts to constrained networks, as it requires only local communication among neighbors, and the transmitted messages are indices from a finite set. We analytically prove the convergence of the proposed algorithm and we show that it outperforms the state-of-the-art greedy methods in terms of balance between recovery accuracy and communication load.

[1]  R.G. Baraniuk,et al.  Distributed Compressed Sensing of Jointly Sparse Signals , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[2]  Simon Foucart,et al.  Hard Thresholding Pursuit: An Algorithm for Compressive Sensing , 2011, SIAM J. Numer. Anal..

[3]  Georgios B. Giannakis,et al.  Distributed Spectrum Sensing for Cognitive Radio Networks by Exploiting Sparsity , 2010, IEEE Transactions on Signal Processing.

[4]  R. Tibshirani The Lasso Problem and Uniqueness , 2012, 1206.0313.

[5]  Cun-Hui Zhang Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[6]  Tong Zhang,et al.  A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems , 2011, 1108.4988.

[7]  H. Zou,et al.  STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION. , 2012, Annals of statistics.

[8]  Enrico Magli,et al.  Distributed ADMM for in-network reconstruction of sparse signals with innovations , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[9]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[10]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[11]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[12]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[13]  Rama Chellappa,et al.  Joint Sparse Representation for Robust Multimodal Biometrics Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Chen Li,et al.  Distributed Compressive Spectrum Sensing in Cooperative Multihop Cognitive Networks , 2011, IEEE Journal of Selected Topics in Signal Processing.

[15]  Xiaojun Chen,et al.  Convergence of the reweighted ℓ1 minimization algorithm for ℓ2–ℓp minimization , 2013, Computational Optimization and Applications.

[16]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[17]  Mikael Skoglund,et al.  Distributed greedy pursuit algorithms , 2013, Signal Process..

[18]  Volkan Cevher,et al.  Combinatorial selection and least absolute shrinkage via the Clash algorithm , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[19]  Stéphane Canu,et al.  Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming , 2009, IEEE Transactions on Signal Processing.

[20]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[21]  Adil M. Bagirov,et al.  Subgradient Method for Nonconvex Nonsmooth Optimization , 2013, J. Optim. Theory Appl..

[22]  Massimo Fornasier,et al.  Numerical Methods for Sparse Recovery , 2010 .

[23]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[24]  Fei Zhou,et al.  Feature Denoising Using Joint Sparse Representation for In-Car Speech Recognition , 2013, IEEE Signal Processing Letters.

[25]  Nannan Yu,et al.  Image Features Extraction and Fusion Based on Joint Sparse Representation , 2011, IEEE Journal of Selected Topics in Signal Processing.

[26]  Volkan Cevher,et al.  Recipes on hard thresholding methods , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[27]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[28]  Jianqing Fan,et al.  Regularization of Wavelet Approximations , 2001 .

[29]  Qing Ling,et al.  Decentralized support detection of multiple measurement vectors with joint sparsity , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[30]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[31]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[32]  Jeffrey D. Blanchard,et al.  Greedy Algorithms for Joint Sparse Recovery , 2014, IEEE Transactions on Signal Processing.

[33]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[34]  Qing Zhou,et al.  Concave penalized estimation of sparse Gaussian Bayesian networks , 2014, J. Mach. Learn. Res..

[35]  Enrico Magli,et al.  Distributed Iterative Thresholding for $\ell _{0}/\ell _{1}$ -Regularized Linear Inverse Problems , 2015, IEEE Transactions on Information Theory.

[36]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[37]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[38]  Jianqing Fan,et al.  Nonconcave Penalized Likelihood With NP-Dimensionality , 2009, IEEE Transactions on Information Theory.

[39]  Pramod K. Varshney,et al.  OMP Based Joint Sparsity Pattern Recovery Under Communication Constraints , 2013, IEEE Transactions on Signal Processing.

[40]  Xiaojun Chen,et al.  Smoothing methods for nonsmooth, nonconvex minimization , 2012, Math. Program..

[41]  David P. Wipf,et al.  Iterative Reweighted 1 and 2 Methods for Finding Sparse Solutions , 2010, IEEE J. Sel. Top. Signal Process..

[42]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[43]  Enrico Magli,et al.  Energy-saving gossip algorithm for compressed sensing in multi-agent systems , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  H. Zou,et al.  One-step Sparse Estimates in Nonconcave Penalized Likelihood Models. , 2008, Annals of statistics.

[45]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[46]  Enrico Magli,et al.  Distributed support detection of jointly sparse signals , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[47]  Enrico Magli,et al.  Randomized Algorithms for Distributed Nonlinear Optimization Under Sparsity Constraints , 2016, IEEE Transactions on Signal Processing.

[48]  Yonina C. Eldar,et al.  Rank Awareness in Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.