From Compressed Sensing to Compressed Bit-Streams: Practical Encoders, Tractable Decoders

Compressed sensing is now established as an effective method for dimension reduction when the underlying signals are sparse or compressible with respect to some suitable basis or frame. One important, yet under-addressed problem regarding the compressive acquisition of analog signals is how to perform quantization. This is directly related to the important issues of how “compressed” compressed sensing is (in terms of the total number of bits one ends up using after acquiring the signal) and ultimately whether compressed sensing can be used to obtain compressed representations of suitable signals. In this paper, we propose a concrete and practicable method for performing “analog-to-information conversion”. Following a compressive signal acquisition stage, the proposed method consists of a quantization stage, based on $ \Sigma \Delta $ (sigma-delta) quantization, and a subsequent encoding (compression) stage that fits within the framework of compressed sensing seamlessly. We prove that, using this method, we can convert analog compressive samples to compressed digital bitstreams and decode using tractable algorithms based on convex optimization. We prove that the proposed analog-to-information converter (AIC) provides a nearly optimal encoding of sparse and compressible signals. Finally, we present numerical experiments illustrating the effectiveness of the proposed AIC.

[1]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[2]  Laurent Jacques,et al.  Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors , 2011, IEEE Transactions on Information Theory.

[3]  Rayan Saab,et al.  Sigma-Delta quantization of sub-Gaussian frame expansions and its application to compressed sensing , 2013, ArXiv.

[4]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[5]  Laurent Jacques,et al.  Stabilizing Nonuniformly Quantized Compressed Sensing With Scalar Companders , 2012, IEEE Transactions on Information Theory.

[6]  Ronald A. DeVore,et al.  Beta expansions: a new approach to digitally corrected A/D conversion , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[7]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[8]  Bernhard G. Bodmann,et al.  Frame paths and error bounds for sigma–delta quantization☆ , 2007 .

[9]  Özgür Yılmaz,et al.  Sobolev Duals in Frame Theory and Sigma-Delta Quantization , 2010 .

[10]  Richard G. Baraniuk,et al.  Quantization of Sparse Representations , 2007, 2007 Data Compression Conference (DCC'07).

[11]  Richard G. Baraniuk,et al.  Theory and Implementation of an Analog-to-Information Converter using Random Demodulation , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[12]  S. Kirolos,et al.  Random Sampling for Analog-to-Information Conversion of Wideband Signals , 2006, 2006 IEEE Dallas/CAS Workshop on Design, Applications, Integration and Software.

[13]  Rongrong Wang,et al.  Quantization of compressive samples with stable and robust recovery , 2015, ArXiv.

[14]  Laurent Jacques,et al.  Quantized Iterative Hard Thresholding: Bridging 1-bit and High-Resolution Quantized Compressed Sensing , 2013, ArXiv.

[15]  I. Daubechies,et al.  Approximating a bandlimited function using very coarsely quantized data: A family of stable sigma-delta modulators of arbitrary order , 2003 .

[16]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[17]  Bernard Chazelle,et al.  The Fast Johnson--Lindenstrauss Transform and Approximate Nearest Neighbors , 2009, SIAM J. Comput..

[18]  Rachel Ward,et al.  New and Improved Johnson-Lindenstrauss Embeddings via the Restricted Isometry Property , 2010, SIAM J. Math. Anal..

[19]  Rayan Saab,et al.  One-Bit Compressive Sensing With Norm Estimation , 2014, IEEE Transactions on Information Theory.

[20]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[21]  Bernard Chazelle,et al.  Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform , 2006, STOC '06.

[22]  Rayan Saab,et al.  Near-Optimal Encoding for Sigma-Delta Quantization of Finite Frame Expansions , 2013, ArXiv.

[23]  Laurent Jacques,et al.  Consistent Basis Pursuit for Signal and Matrix Estimates in Quantized Compressed Sensing , 2015, IEEE Signal Processing Letters.

[24]  Rayan Saab,et al.  Noise-shaping Quantization Methods for Frame-based and Compressive Sampling Systems , 2015, ArXiv.

[25]  Yaniv Plan,et al.  One-bit compressed sensing with non-Gaussian measurements , 2012, ArXiv.

[26]  Yaniv Plan,et al.  One‐Bit Compressed Sensing by Linear Programming , 2011, ArXiv.

[27]  S. Foucart.  Stability and robustness of ℓ1-minimizations with Weibull matrices and redundant dictionaries , 2014 .

[28]  Ashwin A. Wagadarikar,et al.  Single disperser design for coded aperture snapshot spectral imaging. , 2008, Applied optics.

[29]  Jan Vyb'iral.  A variant of the Johnson-Lindenstrauss lemma for circulant matrices , 2010, 1002.2847.

[30]  Rayan Saab,et al.  Root-Exponential Accuracy for Coarse Quantization of Finite Frame Expansions , 2012, IEEE Transactions on Information Theory.

[31]  Rayan Saab,et al.  Sobolev Duals for Random Frames and ΣΔ Quantization of Compressed Sensing Measurements , 2013, Found. Comput. Math..

[32]  John J. Benedetto,et al.  Sigma-delta quantization and finite frames , 2004, ICASSP.

[33]  Rayan Saab,et al.  Sobolev Duals for Random Frames and Sigma-Delta Quantization of Compressed Sensing Measurements , 2010, ArXiv.

[34]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[35]  H. Triebel,et al.  Function Spaces, Entropy Numbers, Differential Operators: Function Spaces , 1996 .

[36]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[37]  Ohad N. Feldheim,et al.  A Universality Result for the Smallest Eigenvalues of Certain Sample Covariance Matrices , 2008, 0812.1961.

[38]  Felix Krahmer,et al.  An optimal family of exponentially accurate one‐bit Sigma‐Delta quantization schemes , 2010, ArXiv.

[39]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[40]  C. S. Güntürk.  One‐bit sigma‐delta quantization with exponential accuracy , 2003 .

[41]  Thomas Kühn,et al.  A Lower Estimate for Entropy Numbers , 2001, J. Approx. Theory.

[42]  Evan Chou,et al.  Beta-Duals of Frames and Applications to Problems in Quantization , 2013 .

[43]  Özgür Yilmaz,et al.  Robust and Practical Analog-to-Digital Conversion With Exponential Precision , 2006, IEEE Transactions on Information Theory.

[44]  R. M. Willett,et al.  Compressed sensing for practical optical imaging systems: A tutorial , 2011, IEEE Photonics Conference 2012.

[45]  Richard G. Baraniuk,et al.  Exponential Decay of Reconstruction Error From Binary Measurements of Sparse Signals , 2014, IEEE Transactions on Information Theory.

[46]  Laurent Jacques,et al.  Quantization and Compressive Sensing , 2014, ArXiv.

[47]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[48]  C. Schütt.  Entropy numbers of diagonal operators between symmetric Banach spaces , 1984 .

[49]  Stephen P. Boyd,et al.  Compressed Sensing With Quantized Measurements , 2010, IEEE Signal Processing Letters.

[50]  Hiroshi Inose,et al.  A unity bit coding method by negative feedback , 1963 .

[51]  Laurent Jacques,et al.  Dequantizing Compressed Sensing: When Oversampling and Non-Gaussian Constraints Combine , 2009, IEEE Transactions on Information Theory.

[52]  Vivek K. Goyal,et al.  Quantized Overcomplete Expansions in IRN: Analysis, Synthesis, and Algorithms , 1998, IEEE Trans. Inf. Theory.