Compressed Counting Meets Compressed Sensing

Compressed sensing (sparse signal recovery) has been a popular and important research topic in recent years. By observing that natural signals are often nonnegative, we propose a new framework for nonnegative signal recovery using Compressed Counting (CC). CC is a technique built on maximally-skewed p-stable random projections originally developed for data stream computations. Our recovery procedure is computationally very efficient in that it requires only one linear scan of the coordinates. Our analysis demonstrates that, when 0 0 and C=pi/2 when p=0.5. In particular, when p->0 the required number of measurements is essentially M=K\log N, where K is the number of nonzero coordinates of the signal.

[1]  I. S. Gradshteyn,et al.  Table of Integrals, Series, and Products , 1976 .

[2]  Frank E. Grubbs,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[3]  C. Mallows,et al.  A Method for Simulating Stable Random Variables , 1976 .

[4]  V. Zolotarev.  One-dimensional stable distributions , 1986 .

[5]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[6]  Noam Nisan,et al.  Pseudorandom generators for space-bounded computations , 1990, STOC '90.

[7]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[8]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[9]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[10]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[11]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[12]  Dan Schnackenberg,et al.  Statistical approaches to DDoS attack detection and response , 2003, Proceedings DARPA Information Survivability Conference and Exposition.

[13]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[14]  Graham Cormode,et al.  An improved data stream summary: the count-min sketch and its applications , 2004, J. Algorithms.

[15]  E. Candes,et al.  11-magic : Recovery of sparse signals via convex programming , 2005 .

[16]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[17]  Piotr Indyk,et al.  Stable distributions, pseudorandom generators, embeddings, and data stream computation , 2006, JACM.

[18]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[19]  Ping Li,et al.  Very sparse stable random projections for dimension reduction in lα (0 <α ≤ 2) norm , 2007, KDD '07.

[20]  Ashwin Lall,et al.  A data streaming algorithm for estimating entropies of od flows , 2007, IMC '07.

[21]  Ping Li,et al.  Estimators and tail bounds for dimension reduction in lα (0 < α ≤ 2) using stable random projections , 2008, SODA '08.

[22]  Ping Li,et al.  The Optimal Quantile Estimator for Compressed Counting , 2008, ArXiv.

[23]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[24]  Ping Li,et al.  On Approximating Frequency Moments of Data Streams with Skewed Projections , 2008, ArXiv.

[25]  S. Rangan,et al.  Orthogonal Matching Pursuit from Noisy Measurements : A New Analysis ∗ , 2009 .

[26]  Ping Li,et al.  Compressed counting , 2008, SODA.

[27]  Sundeep Rangan,et al.  Orthogonal Matching Pursuit From Noisy Random Measurements: A New Analysis , 2009, NIPS.

[28]  Ping Li,et al.  Improving Compressed Counting , 2009, UAI.

[29]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Matrices , 2010, Proceedings of the IEEE.

[30]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[31]  Ping Li,et al.  A New Algorithm for Compressed Counting with Applications in Shannon Entropy Estimation in Dynamic Data , 2011, COLT.

[32]  H. T. Kung,et al.  Compressive sensing medium access control for wireless LANs , 2012, 2012 IEEE Global Communications Conference (GLOBECOM).

[33]  Ping Li,et al.  Entropy Estimations Using Correlated Symmetric Stable Random Projections , 2012, NIPS.

[34]  Enrique Mallada,et al.  Sparse recovery with graph constraints: Fundamental limits and measurement construction , 2011, 2012 Proceedings IEEE INFOCOM.

[35]  Piotr Indyk,et al.  Efficient and reliable low-power backscatter networks , 2012, CCRV.

[36]  Ping Li,et al.  Exact sparse recovery with L0 projections , 2013, KDD.

[37]  Ping Li,et al.  Sparse Recovery with Very Sparse Compressed Counting , 2014, ArXiv.

[38]  Ping Li,et al.  Coding for Random Projections , 2013, ICML.

[39]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[40]  Kerstin Vogler,et al.  Table Of Integrals Series And Products , 2016 .