Distributed Compressed Sensing Dror

Compressed sensing is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for reconstruction. In this paper we introduce a new theory for distributed compressed sensing (DCS) that enables new distributed coding algorithms for multi-signal ensembles that exploit both intraand inter-signal correlation structures. The DCS theory rests on a new concept that we term the joint sparsity of a signal ensemble. We study in detail three simple models for jointly sparse signals, propose algorithms for joint recovery of multiple signals from incoherent projections, and characterize theoretically and empirically the number of measurements per sensor required for accurate reconstruction. We establish a parallel with the Slepian-Wolf theorem from information theory and establish upper and lower bounds on the measurement rates required for encoding jointly sparse signals. In two of our three models, the results are asymptotically best-possible, meaning that both the upper and lower bounds match the performance of our practical algorithms. Moreover, simulations indicate that the asymptotics take effect with just a moderate number of signals. In some sense DCS is a framework for distributed compression of sources with memory, which has remained a challenging problem for some time. DCS is immediately applicable to a range of problems in sensor networks and arrays.

[1]  Michelle Effros,et al.  Universal lossless source coding with the Burrows Wheeler transform , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[2]  J. Tropp Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[3]  Richard G. Baraniuk,et al.  Analysis of the DCS one-stage Greedy Algorothm for Common Sparse Supports , 2005 .

[4]  Gregory J. Pottie,et al.  Wireless integrated network sensors , 2000, Commun. ACM.

[5]  Antonio Ortega,et al.  A distributed wavelet compression algorithm for wireless multihop sensor networks using lifting , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[6]  R.G. Baraniuk,et al.  Distributed Compressed Sensing of Jointly Sparse Signals , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[7]  Joan L. Mitchell,et al.  JPEG: Still Image Data Compression Standard , 1992 .

[8]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[9]  Baltasar Beferull-Lozano,et al.  On network correlated data gathering , 2004, IEEE INFOCOM 2004.

[10]  Joel A. Tropp,et al.  Simultaneous sparse approximation via greedy pursuit , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[11]  Justin Romberg,et al.  Practical Signal Recovery from Random Projections , 2005 .

[12]  Yoram Bresler,et al.  Further results on spectrum blind sampling of 2D signals , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[13]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[14]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[15]  Jean-Jacques Fuchs,et al.  Recovery of exact sparse representations in the presence of bounded noise , 2005, IEEE Transactions on Information Theory.

[16]  Hyeokho Choi,et al.  Distributed wavelet transform for irregular sensor network grids , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[17]  Michael Gastpar,et al.  The Distributed Karhunen–Loève Transform , 2006, IEEE Transactions on Information Theory.

[18]  R.G. Baraniuk,et al.  Universal distributed sensing via random projections , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[19]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[20]  Deborah Estrin,et al.  An evaluation of multi-resolution storage for sensor networks , 2003, SenSys '03.

[21]  Emmanuel J. Candès,et al.  Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions , 2004, Found. Comput. Math..

[22]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[25]  E. Candès,et al.  Error correction via linear programming , 2005, FOCS 2005.

[26]  Richard G. Baraniuk,et al.  An Information-Theoretic Approach to Distributed Compressed Sensing ∗ , 2005 .

[27]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[28]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[29]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[30]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[31]  D. Takhar,et al.  A compressed sensing camera : New theory and an implementation using digital micromirrors , 2006 .

[32]  K. Ramchandran,et al.  Distributed source coding using syndromes (DISCUS): design and construction , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[33]  Thomas M. Cover,et al.  A Proof of the Data Compression Theorem of Slepian and Wolf for Ergodic Sources , 1971 .

[34]  Zixiang Xiong,et al.  Distributed source coding for sensor networks , 2004, IEEE Signal Processing Magazine.

[35]  R. A. McDonald,et al.  Noiseless Coding of Correlated Information Sources , 1973 .

[36]  Michael T. Orchard,et al.  Space-frequency quantization for wavelet image coding , 1996, Optics & Photonics.

[37]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[38]  Kannan Ramchandran,et al.  PRISM: A new robust video coding architecture based on distributed compression principles , 2002 .

[39]  Tomohiko Uyematsu,et al.  Universal Coding for Correlated Sources with Memory , 2001 .

[40]  Gaurav S. Sukhatme,et al.  Connecting the Physical World with Pervasive Networks , 2002, IEEE Pervasive Comput..

[41]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[42]  Deborah Estrin,et al.  Modelling Data-Centric Routing in Wireless Sensor Networks , 2002 .

[43]  V. Temlyakov A remark on simultaneous greedy approximation , 2004 .

[44]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Yoram Bresler,et al.  An O(N) semipredictive universal encoder via the BWT , 2004, IEEE Transactions on Information Theory.

[46]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[47]  Minh N. Do,et al.  Signal reconstruction using sparse tree representations , 2005, SPIE Optics + Photonics.

[48]  Richard G. Baraniuk,et al.  Fast reconstruction of piecewise smooth signals from random projections , 2005 .

[49]  Robert D. Nowak,et al.  Distributed image compression for sensor networks using correspondence analysis and super-resolution , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[50]  Ronald A. DeVore,et al.  Image compression through wavelet transform coding , 1992, IEEE Trans. Inf. Theory.

[51]  Gregory J. Pottie,et al.  Routing Explicit Side Information for Data Compression in Wireless Sensor Networks , 2005, DCOSS.

[52]  Karlheinz Brandenburg,et al.  MP3 and AAC Explained , 1999 .

[53]  Richard G. Baraniuk,et al.  Recovery of Jointly Sparse Signals from Few Random Projections , 2005, NIPS.

[54]  S. Mallat A wavelet tour of signal processing , 1998 .