Modified distributed iterative hard thresholding

In this paper, we suggest a modified distributed compressed sensing (CS) approach based on the iterative hard thresholding (IHT) algorithm, namely, distributed IHT (DIHT). Our technique improves upon a recently proposed DIHT algorithm in two ways. First, for sensing matrices with i.i.d. Gaussian entries, we suggest an efficient and tight method for computing the step size μ in IHT based on random matrix theory. Second, we improve upon the global computation (GC) step of DIHT by adapting this step to allow for complex data, and reducing the communication cost. The new GC operation involves solving a Top-K problem and is therefore referred to as GC.K. The GC.K-based DIHT has exactly the same recovery results as the centralized IHT given the same step size μ. Numerical results show that our approach significantly outperforms the modified thresholding algorithm (MTA), another GC algorithm for DIHT proposed in previous work. Our simulations also verify that the proposed method of computing μ renders the performance of DIHT close to the oracle-aided approach with a given “optimal” μ.

[1]  Moni Naor,et al.  Optimal aggregation algorithms for middleware , 2001, PODS '01.

[2]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[3]  Zhe Wang,et al.  Efficient top-K query calculation in distributed networks , 2004, PODC '04.

[4]  Sergios Theodoridis,et al.  A greedy sparsity-promoting LMS for distributed adaptive learning in diffusion networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Sergios Theodoridis,et al.  A Sparsity Promoting Adaptive Algorithm for Distributed Learning , 2012, IEEE Transactions on Signal Processing.

[6]  E.J. Candes.  Compressive Sampling , 2022 .

[7]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[8]  I. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[9]  Ali H. Sayed,et al.  Sparse Distributed Learning Based on Diffusion Adaptation , 2012, IEEE Transactions on Signal Processing.

[10]  Yonina C. Eldar,et al.  Distributed Compressed Sensing for Static and Time-Varying Networks , 2013, IEEE Transactions on Signal Processing.

[11]  Mikael Skoglund,et al.  A greedy pursuit algorithm for distributed compressed sensing , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[13]  Ali H. Sayed,et al.  Online dictionary learning over distributed models , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[15]  João M. F. Xavier,et al.  Distributed Basis Pursuit , 2010, IEEE Transactions on Signal Processing.

[16]  Yonina C. Eldar,et al.  Distributed approximate message passing for sparse signal recovery , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[17]  Yonina C. Eldar,et al.  Distributed sparse signal recovery for sensor networks , 2012, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.