Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors

The compressive sensing (CS) framework aims to ease the burden on analog-to-digital converters (ADCs) by reducing the sampling rate required to acquire and stably recover sparse signals. Practical ADCs not only sample but also quantize each measurement to a finite number of bits; moreover, there is an inverse relationship between the achievable sampling rate and the bit depth. In this paper, we investigate an alternative CS approach that shifts the emphasis from the sampling rate to the number of bits per measurement. In particular, we explore the extreme case of 1-bit CS measurements, which capture just their sign. Our results come in two flavors. First, we consider ideal reconstruction from noiseless 1-bit measurements and provide a lower bound on the best achievable reconstruction error. We also demonstrate that i.i.d. random Gaussian matrices provide measurement mappings that, with overwhelming probability, achieve nearly optimal error decay. Next, we consider reconstruction robustness to measurement errors and noise and introduce the binary ε-stable embedding property, which characterizes the robustness of the measurement process to sign changes. We show that the same class of matrices that provide almost optimal noiseless performance also enable such a robust mapping. On the practical side, we introduce the binary iterative hard thresholding algorithm for signal reconstruction from 1-bit measurements that offers state-of-the-art performance.

[1]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[2]  L. Flatto A NEW PROOF OF THE TRANSPOSITION THEOREM , 1970 .

[3]  G. Zeoli,et al.  Quantization and Saturation Noise Due to Analog-to-Digital Conversion , 1971, IEEE Transactions on Aerospace and Electronic Systems.

[4]  P. J. Huber Robust Regression: Asymptotics, Conjectures and Monte Carlo , 1973 .

[5]  Gabor C. Temes,et al.  Oversampling delta-sigma data converters : theory, design, and simulation , 1992 .

[6]  F. Famoye Continuous Univariate Distributions, Volume 1 , 1994 .

[7]  Martin Vetterli,et al.  Reduction of the MSE in R-times oversampled A/D conversion O(1/R) to O(1/R2) , 1994, IEEE Trans. Signal Process..

[8]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[9]  Vivek K Goyal Quantized Overcomplete Expansions : Analysis , Synthesis and Algorithms , 1995 .

[10]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[11]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[12]  H. V. Sorensen,et al.  An overview of sigma-delta converters , 1996, IEEE Signal Process. Mag..

[13]  G. Lorentz,et al.  Constructive approximation : advanced problems , 1996 .

[14]  Nguyen T. Thao Vector quantization analysis of ΣΔ modulation , 1996, IEEE Trans. Signal Process..

[15]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  Avrim Blum,et al.  On-line Algorithms in Machine Learning , 1996, Online Algorithms.

[17]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[18]  K. Ball An elementary introduction to modern convex geometry, in flavors of geometry , 1997 .

[19]  K. Ball An Elementary Introduction to Modern Convex Geometry , 1997 .

[20]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[21]  Vivek K. Goyal,et al.  Quantized Overcomplete Expansions in IRN: Analysis, Synthesis, and Algorithms , 1998, IEEE Trans. Inf. Theory.

[22]  Jeffrey Scott Vitter,et al.  Proceedings of the thirtieth annual ACM symposium on Theory of computing , 1998, STOC 1998.

[23]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[24]  Robert H. Walden,et al.  Analog-to-digital converter survey and analysis , 1999, IEEE J. Sel. Areas Commun..

[25]  Susanne Albers,et al.  On‐Line Algorithms , 2013 .

[26]  Thomas Strohmer,et al.  GRASSMANNIAN FRAMES WITH APPLICATIONS TO CODING AND COMMUNICATION , 2003, math/0301135.

[27]  C. S. Güntürk One‐bit sigma‐delta quantization with exponential accuracy , 2003 .

[28]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[29]  Lorenzo Rosasco,et al.  Are Loss Functions All the Same? , 2004, Neural Computation.

[30]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[31]  John J. Benedetto,et al.  Sigma-delta quantization and finite frames , 2004, ICASSP.

[32]  J. Benedetto,et al.  Sigma-delta (/spl Sigma//spl Delta/) quantization and finite frames , 2006, IEEE Transactions on Information Theory.

[33]  C.W. Bostian,et al.  Analog-to-digital converters , 2005, IEEE Signal Processing Magazine.

[34]  Gunnar Rätsch,et al.  Efficient Margin Maximizing with Boosting , 2005, J. Mach. Learn. Res..

[35]  Alan V. Oppenheim,et al.  Quantization noise shaping on arbitrary frame expansions , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[36]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[37]  E.J. Candes Compressive Sampling , 2022 .

[38]  Alexandr Andoni,et al.  Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[39]  Richard G. Baraniuk,et al.  Random Filters for Compressive Sampling and Reconstruction , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[40]  A. Oppenheim,et al.  Quantization and erasures in frame representations , 2006 .

[41]  Wotao Yin,et al.  TR 0707 A Fixed-Point Continuation Method for ` 1-Regularized Minimization with Applications to Compressed Sensing , 2007 .

[42]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[43]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[44]  Richard G. Baraniuk,et al.  Sigma delta quantization for compressive sensing , 2007, SPIE Optical Engineering + Applications.

[45]  Richard G. Baraniuk,et al.  Quantization of Sparse Representations , 2007, 2007 Data Compression Conference (DCC'07).

[46]  M. Bridson Geometric and Combinatorial Group Theory , 2007 .

[47]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[48]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[49]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[50]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[51]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[52]  Justin K. Romberg,et al.  Compressive Sensing by Random Convolution , 2009, SIAM J. Imaging Sci..

[53]  Svetlana Lazebnik,et al.  Locality-sensitive binary codes from shift-invariant kernels , 2009, NIPS.

[54]  Vivek K. Goyal,et al.  Optimal quantization of random measurements in compressed sensing , 2009, 2009 IEEE International Symposium on Information Theory.

[55]  Vivek K Goyal,et al.  Quantization for Compressed Sensing Reconstruction , 2009 .

[56]  Laurent Jacques,et al.  CMOS compressed imaging by Random Convolution , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[57]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[58]  Olgica Milenkovic,et al.  Distortion-rate functions for quantized compressive sensing , 2009, 2009 IEEE Information Theory Workshop on Networking and Information Theory.

[59]  P. Boufounos Greedy sparse signal reconstruction from sign measurements , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[60]  Laurent Jacques,et al.  Randomly driven fuzzy key extraction of unclonable images , 2010, 2010 IEEE International Conference on Image Processing.

[61]  Ambuj Tewari,et al.  Smoothness, Low Noise and Fast Rates , 2010, NIPS.

[62]  T. Blumensath Compressed Sensing with Nonlinear Observations , 2010 .

[63]  Stephen P. Boyd,et al.  Compressed Sensing With Quantized Measurements , 2010, IEEE Signal Processing Letters.

[64]  Vivek K. Goyal,et al.  Frame permutation quantization , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[65]  Mark A. Davenport,et al.  Random Observations on Random Observations: Sparse Signal Acquisition and Processing , 2010 .

[66]  Petros Boufounos,et al.  Reconstruction of sparse signals from distorted randomized measurements , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[67]  Robert D. Nowak,et al.  Sample complexity for 1-bit compressed sensing and sparse classification , 2010, 2010 IEEE International Symposium on Information Theory.

[68]  Justin K. Romberg,et al.  Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals , 2009, IEEE Transactions on Information Theory.

[69]  Rayan Saab,et al.  Sobolev Duals for Random Frames and Sigma-Delta Quantization of Compressed Sensing Measurements , 2010, ArXiv.

[70]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[71]  Laurent Jacques,et al.  Dequantizing Compressed Sensing: When Oversampling and Non-Gaussian Constraints Combine , 2009, IEEE Transactions on Information Theory.

[72]  Richard G. Baraniuk,et al.  Democracy in Action: Quantization, Saturation, and Compressive Sensing , 2011 .

[73]  P. Boufounos Hierarchical Distributed Scalar Quantization , 2011 .

[74]  Wotao Yin,et al.  Trust, But Verify: Fast and Accurate Signal Recovery From 1-Bit Compressive Measurements , 2011, IEEE Transactions on Signal Processing.

[75]  Yaniv Plan,et al.  One‐Bit Compressed Sensing by Linear Programming , 2011, ArXiv.

[76]  Piotr Indyk,et al.  Approximate Nearest Neighbor: Towards Removing the Curse of Dimensionality , 2012, Theory Comput..

[77]  Petros Boufounos,et al.  Universal Rate-Efficient Scalar Quantization , 2010, IEEE Transactions on Information Theory.

[78]  Yaniv Plan,et al.  Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach , 2012, IEEE Transactions on Information Theory.

[79]  Yaniv Plan,et al.  Dimension Reduction by Random Hyperplane Tessellations , 2014, Discret. Comput. Geom..

[80]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.