Robustness to Unknown Error in Sparse Regularization

Quadratically constrained basis pursuit has become a popular device in sparse regularization; in particular, in the context of compressed sensing. However, the majority of theoretical error estimates for this regularizer assume an a priori bound on the noise level, which is usually lacking in practice. In this paper, we develop stability and robustness estimates, which remove this assumption. First, we introduce an abstract framework and show that the robust instance optimality of any decoder in the noise-aware setting implies stability and robustness in the noise-blind setting. This is based on certain sup-inf constants referred to as quotients, strictly related to the quotient property of compressed sensing. Then, we apply this theory to prove the robustness of quadratically constrained basis pursuit under unknown error in the cases of random Gaussian matrices and of random matrices with heavy-tailed rows, such as random sampling matrices from bounded orthonormal systems. We illustrate our results in several cases of practical importance, including subsampled Fourier measurements and the recovery of sparse polynomial expansions.

[1]  Ben Adcock,et al.  Infinite-Dimensional Compressed Sensing and Function Interpolation , 2015, Foundations of Computational Mathematics.

[2]  Yaniv Plan,et al.  The Generalized Lasso With Non-Linear Observations , 2015, IEEE Transactions on Information Theory.

[3]  B. Logan,et al.  Signal recovery and the large sieve , 1992 .

[4]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[5]  H. Rauhut Compressive Sensing and Structured Random Matrices , 2009 .

[6]  Xiu Yang,et al.  Reweighted ℓ1ℓ1 minimization method for stochastic elliptic differential equations , 2013, J. Comput. Phys..

[7]  Ben Adcock,et al.  Compressed Sensing Approaches for Polynomial Approximation of High-Dimensional Functions , 2017, 1703.06987.

[8]  H. Rauhut,et al.  Compressive sensing Petrov-Galerkin approximation of high-dimensional parametric operator equations , 2014, Mathematics of Computation.

[9]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[10]  Ben Adcock,et al.  Infinite-dimensional $\ell^1$ minimization and function approximation from pointwise data , 2015, 1503.02352.

[11]  Simona Perotto,et al.  Compressed solving: A numerical approximation technique for elliptic PDEs based on Compressed Sensing , 2015, Comput. Math. Appl..

[12]  Houman Owhadi,et al.  A non-adapted sparse approximation of PDEs with stochastic inputs , 2010, J. Comput. Phys..

[13]  Holger Rauhut,et al.  Compressive sensing Petrov-Galerkin approximation of high-dimensional parametric operator equations , 2014, Math. Comput..

[14]  Ben Adcock,et al.  Recovery guarantees for Compressed Sensing with unknown errors , 2017, 2017 International Conference on Sampling Theory and Applications (SampTA).

[15]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[16]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[17]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[18]  Oded Regev,et al.  The Restricted Isometry Property of Subsampled Fourier Matrices , 2015, SODA.

[19]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[20]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[21]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[22]  Guannan Zhang,et al.  Analysis of quasi-optimal polynomial approximations for parameterized PDEs with deterministic and stochastic coefficients , 2015, Numerische Mathematik.

[23]  Ben Adcock,et al.  On asymptotic structure in compressed sensing , 2014, ArXiv.

[24]  Holger Rauhut,et al.  Compressed sensing Petrov-Galerkin approximations for parametric PDEs , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[25]  Hoang Tran,et al.  Polynomial approximation via compressed sensing of high-dimensional functions on lower sets , 2016, Math. Comput..

[26]  Klaas Paul Pruessmann,et al.  Realistic Analytical Phantoms for Parallel Magnetic Resonance Imaging , 2012, IEEE Transactions on Medical Imaging.

[27]  S. Foucart Stability and robustness of ℓ1-minimizations with Weibull matrices and redundant dictionaries , 2014 .

[28]  Holger Rauhut,et al.  Compressive Sensing with structured random matrices , 2012 .

[29]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[30]  Martin J. Wainwright,et al.  A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers , 2009, NIPS.

[31]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[32]  AdcockBen,et al.  Generalized Sampling and Infinite-Dimensional Compressed Sensing , 2016 .

[33]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[34]  Massimo Fornasier,et al.  Theoretical Foundations and Numerical Methods for Sparse Recovery , 2010, Radon Series on Computational and Applied Mathematics.

[35]  H. Rauhut,et al.  Interpolation via weighted $l_1$ minimization , 2013, 1308.0759.

[36]  P. Wojtaszczyk,et al.  Stability and Instance Optimality for Gaussian Measurements in Compressed Sensing , 2010, Found. Comput. Math..

[37]  Thomas Strohmer,et al.  General Deviants: An Analysis of Perturbations in Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[38]  Simona Perotto,et al.  A theoretical study of COmpRessed SolvING for advection-diffusion-reaction problems , 2017, Math. Comput..

[39]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[40]  Anru Zhang,et al.  Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-Rank Matrices , 2013, IEEE Transactions on Information Theory.

[41]  J. Bourgain An Improved Estimate in the Restricted Isometry Problem , 2014 .

[42]  R. DeVore,et al.  Instance-optimality in probability with an ℓ1-minimization decoder , 2009 .

[43]  Ewout van den Berg,et al.  1-Bit Matrix Completion , 2012, ArXiv.