Dequantizing Compressed Sensing: When Oversampling and Non-Gaussian Constraints Combine

In this paper, we study the problem of recovering sparse or compressible signals from uniformly quantized measurements. We present a new class of convex optimization programs, or decoders, coined Basis Pursuit DeQuantizer of moment p (BPDQp), that model the quantization distortion more faithfully than the commonly used Basis Pursuit DeNoise (BPDN) program. Our decoders proceed by minimizing the sparsity of the signal to be reconstructed subject to a data-fidelity constraint expressed in the ℓp-norm of the residual error for 2 ≤ p ≤ ∞. We show theoretically that, (i) the reconstruction error of these new decoders is bounded if the sensing matrix satisfies an extended Restricted Isometry Property involving the Iρ norm, and (ii), for Gaussian random matrices and uniformly quantized measurements, BPDQp performance exceeds that of BPDN by dividing the reconstruction error due to quantization by √(p + 1). This last effect happens with high probability when the number of measurements exceeds a value growing with p, i.e., in an oversampled situation compared to what is commonly required by BPDN = BPDQ2. To demonstrate the theoretical power of BPDQp, we report numerical simulations on signal and image reconstruction problems.

[1]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[2]  Rémi Gribonval,et al.  Restricted Isometry Constants Where $\ell ^{p}$ Sparse Recovery Can Fail for $0≪ p \leq 1$ , 2009, IEEE Transactions on Information Theory.

[3]  V.K. Goyal,et al.  Compressive Sampling and Lossy Compression , 2008, IEEE Signal Processing Magazine.

[4]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[5]  Pierre Vandergheynst,et al.  Compressed Sensing and Redundant Dictionaries , 2007, IEEE Transactions on Information Theory.

[6]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[7]  Emmanuel J. Candès,et al.  Encoding the /spl lscr//sub p/ ball from limited measurements , 2006, Data Compression Conference (DCC'06).

[8]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Matrices , 2010, Proceedings of the IEEE.

[9]  Hong-Kun Xu.  Inequalities in Banach spaces with applications , 1991 .

[10]  Restricted Isometry Constants where p sparse , 2011 .

[11]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[12]  Laurent Jacques,et al.  DeQuantizing Compressed Sensing with non-Gaussian constraints , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[13]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[14]  M. Varanasi,et al.  Parametric generalized Gaussian density estimation , 1989 .

[15]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[16]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[17]  Martin Vetterli,et al.  Deterministic analysis of oversampled A/D conversion and decoding improvement based on consistent estimates , 1994, IEEE Trans. Signal Process..

[18]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[19]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[20]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[21]  R. Spira.  Calculation of the Gamma Function by Stirling's Formula , 1971 .

[22]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[23]  Olgica Milenkovic,et al.  Quantized Compressive Sensing , 2009, 0901.0749.

[24]  Richard G. Baraniuk,et al.  Exact signal recovery from sparsely corrupted measurements through the Pursuit of Justice , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[25]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[26]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[27]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[28]  Daniel Zwillinger,et al.  CRC standard mathematical tables and formulae; 30th edition , 1995 .

[29]  Richard G. Baraniuk,et al.  Quantization of Sparse Representations , 2007, 2007 Data Compression Conference (DCC'07).

[30]  R. D. Carmichael,et al.  Mathematical tables and formulae , 1958 .

[31]  Emmanuel J. Cand.  The Restricted Isometry Property and Its Implications for Compressed Sensing , 2008 .

[32]  Jean-Jacques Fuchs,et al.  Fast implementation of a ℓ1 - ℓ1 regularized sparse representations algorithm. , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Davies Rémi Gribonval.  Restricted Isometry Constants Where Lp Sparse Recovery Can Fail for 0 , 2008 .

[34]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[35]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[36]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[37]  Stephen P. Boyd,et al.  Compressed Sensing With Quantized Measurements , 2010, IEEE Signal Processing Letters.

[38]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[39]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[40]  Marc Antonini,et al.  Compression artifacts reduction using variational methods : Algorithms and experimental study , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[41]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[42]  Richard G. Baraniuk,et al.  Democracy in Action: Quantization, Saturation, and Compressive Sensing , 2011 .

[43]  E. Candès,et al.  Encoding the ` p Ball from Limited Measurements , 2006 .

[44]  W. L. Bynum,et al.  Weak Parallelogram Laws for Banach Spaces , 1976, Canadian Mathematical Bulletin.

[45]  W. Beyer.  CRC Standard Mathematical Tables and Formulae , 1991 .

[46]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[47]  Curtis R. Vogel,et al.  Iterative Methods for Total Variation Denoising , 1996, SIAM J. Sci. Comput..

[48]  S. Mendelson,et al.  Reconstruction and Subgaussian Operators in Asymptotic Geometric Analysis , 2007 .

[49]  Olgica Milenkovic,et al.  Distortion-rate functions for quantized compressive sensing , 2009, 2009 IEEE Information Theory Workshop on Networking and Information Theory.

[50]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[51]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[52]  Leslie Ying,et al.  Linear transformations and Restricted Isometry Property , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[53]  Michel Verleysen,et al.  The Concentration of Fractional Distances , 2007, IEEE Transactions on Knowledge and Data Engineering.

[54]  Emmanuel J. Candès,et al.  Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions , 2004, Found. Comput. Math..