Regime Change: Bit-Depth Versus Measurement-Rate in Compressive Sensing

The recently introduced compressive sensing (CS) framework enables digital signal acquisition systems to take advantage of signal structures beyond bandlimitedness. Indeed, the number of CS measurements required for stable reconstruction is closer to the order of the signal complexity than the Nyquist rate. To date, the CS theory has focused on real-valued measurements, but in practice measurements are mapped to bits from a finite alphabet. Moreover, in many potential applications the total number of measurement bits is constrained, which suggests a tradeoff between the number of measurements and the number of bits per measurement. We study this situation in this paper and show that there exist two distinct regimes of operation that correspond to high/low signal-to-noise ratio (SNR). In the measurement compression (MC) regime, a high SNR favors acquiring fewer measurements with more bits per measurement; in the quantization compression (QC) regime, a low SNR favors acquiring more measurements with fewer bits per measurement. A surprise from our analysis and experiments is that in many practical applications it is better to operate in the QC regime, even acquiring as few as 1 bit per measurement.

[1]  E.J. Candes.  Compressive Sampling , 2022 .

[2]  Richard G. Baraniuk,et al.  The Pros and Cons of Compressive Sensing for Wideband Signal Acquisition: Noise Folding versus Dynamic Range , 2011, IEEE Transactions on Signal Processing.

[3]  Stephen J. Wright,et al.  Toeplitz-Structured Compressed Sensing Matrices , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[4]  Yaniv Plan,et al.  One‐Bit Compressed Sensing by Linear Programming , 2011, ArXiv.

[5]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[6]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[7]  Marco F. Duarte,et al.  Compressive sensing recovery of spike trains using a structured sparsity model , 2009 .

[8]  Laurent Jacques,et al.  Dequantizing Compressed Sensing: When Oversampling and Non-Gaussian Constraints Combine , 2009, IEEE Transactions on Information Theory.

[9]  Vivek K. Goyal,et al.  Quantized Overcomplete Expansions in IRN: Analysis, Synthesis, and Algorithms , 1998, IEEE Trans. Inf. Theory.

[10]  Mike E. Davies,et al.  Sample-distortion functions for compressed sensing , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[11]  Brian M. Sadler,et al.  Monobit digital receivers for ultrawideband communications , 2005, IEEE Transactions on Wireless Communications.

[12]  Emmanuel J. Candès,et al.  How well can we estimate a sparse vector? , 2011, ArXiv.

[13]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[14]  D. L. Donoho,et al.  Rapid MR Imaging with "Compressed Sensing" and Randomly Under-Sampled 3DFT Trajectories , 2004 .

[15]  David L. Neuhoff,et al.  Quantization , 2022, IEEE Trans. Inf. Theory.

[16]  A. Oppenheim,et al.  Quantization and erasures in frame representations , 2006 .

[17]  Gabor C. Temes,et al.  Oversampling delta-sigma data converters : theory, design, and simulation , 1992 .

[18]  Rayan Saab,et al.  Sobolev Duals for Random Frames and Sigma-Delta Quantization of Compressed Sensing Measurements , 2010, ArXiv.

[19]  Richard G. Baraniuk,et al.  Democracy in Action: Quantization, Saturation, and Compressive Sensing , 2011 .

[20]  Rayan Saab,et al.  Sigma delta quantization for compressed sensing , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[21]  Wotao Yin,et al.  TR 0707 A Fixed-Point Continuation Method for ` 1-Regularized Minimization with Applications to Compressed Sensing , 2007 .

[22]  Ingrid Daubechies,et al.  Single-Bit Oversampled A/D Conversion With Exponential Accuracy in the Bit Rate , 2007, IEEE Transactions on Information Theory.

[23]  Wotao Yin,et al.  Trust, But Verify: Fast and Accurate Signal Recovery From 1-Bit Compressive Measurements , 2011, IEEE Transactions on Signal Processing.

[24]  Richard G. Baraniuk,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[25]  Vivek K Goyal.  Quantized Overcomplete Expansions : Analysis , Synthesis and Algorithms , 1995 .

[26]  John J. Benedetto,et al.  Sigma-delta quantization and finite frames , 2004, ICASSP.

[27]  Laurent Jacques,et al.  Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors , 2011, IEEE Transactions on Information Theory.

[28]  H. V. Sorensen,et al.  An overview of sigma-delta converters , 1996, IEEE Signal Process. Mag..

[29]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[30]  John J. Benedetto,et al.  Sigma-delta (/spl Sigma//spl Delta/) quantization and finite frames , 2006, IEEE Trans. Inf. Theory.

[31]  Vivek K. Goyal,et al.  Frame permutation quantization , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[32]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[33]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[34]  Richard G. Baraniuk,et al.  The compressive multiplexer for multi-channel compressive sensing , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[36]  R. Feynman.  There’s plenty of room at the bottom , 2011 .

[37]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[38]  C.W. Bostian,et al.  Analog-to-digital converters , 2005, IEEE Signal Processing Magazine.

[39]  Vivek K. Goyal,et al.  Optimal quantization of random measurements in compressed sensing , 2009, 2009 IEEE International Symposium on Information Theory.

[40]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[41]  Stephen P. Boyd,et al.  Compressed Sensing With Quantized Measurements , 2010, IEEE Signal Processing Letters.

[42]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[43]  P. Boufounos.  Greedy sparse signal reconstruction from sign measurements , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[44]  Richard Baraniuk,et al.  APPLICATION OF COMPRESSIVE SENSING TO THE DESIGN OF WIDEBAND SIGNAL ACQUISITION RECEIVERS , 2009 .

[45]  Vivek K Goyal,et al.  Quantization for Compressed Sensing Reconstruction , 2009 .

[46]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[47]  Petros Boufounos,et al.  Universal Rate-Efficient Scalar Quantization , 2010, IEEE Transactions on Information Theory.

[48]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[49]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[50]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[51]  Linhui Liu,et al.  A methodology for the simulation of digital-to-analog converters (Withdrawal Notice) , 2009, International Conference on Photonics and Image in Agriculture Engineering.

[52]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[53]  Marco F. Duarte,et al.  Spectral compressive sensing , 2013 .