Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems

In this paper, we are concerned with compressive sampling strategies and sparse recovery principles for linear inverse and ill-posed problems. As the main result, we provide compressed measurement models for ill-posed problems and recovery accuracy estimates for sparse approximations of the solution of the underlying inverse problem. The main ingredients are variational formulations that allow the treatment of ill-posed operator equations in the context of compressively sampled data. In particular, we rely on Tikhonov variational and constrained optimization formulations. One essential difference to the classical compressed sensing framework is the incorporation of joint sparsity measures allowing the treatment of infinite-dimensional reconstruction spaces. The theoretical results are furnished with a number of numerical experiments.

[1]  Karsten Urban,et al.  Adaptive Wavelet Methods on Unbounded Domains , 2012, Journal of Scientific Computing.

[2]  Armin Iske,et al.  Optimally Sparse Image Representation by the Easy Path Wavelet Transform , 2012, Int. J. Wavelets Multiresolution Inf. Process..

[3]  Winfried Sickel,et al.  Best m-Term Approximation and Sobolev–Besov Spaces of Dominating Mixed Smoothness—the Case of Compact Embeddings , 2012 .

[4]  Gerlind Plonka-Hoch,et al.  Curvelet-Wavelet Regularized Split Bregman Iteration for Compressed Sensing , 2011, Int. J. Wavelets Multiresolution Inf. Process..

[5]  G. Teschke,et al.  Inversion of the noisy Radon transform on SO(3) by Gabor frames and sparse recovery principles , 2011 .

[6]  Harry Yserentant,et al.  The mixed regularity of electronic wave functions multiplied by explicit correlation factors , 2011 .

[7]  Winfried Sickel,et al.  Best m-term approximation and Lizorkin-Triebel spaces , 2011, J. Approx. Theory.

[8]  Steffen Dereich,et al.  A multilevel Monte Carlo algorithm for Lévy-driven stochastic differential equations , 2011 .

[9]  Fred J. Hickernell,et al.  Deterministic multi-level algorithms for infinite-dimensional integration on RN , 2011, J. Complex..

[10]  P. Maass,et al.  Function spaces and optimal currents in impedance tomography , 2011 .

[11]  Lutz Kämmerer,et al.  On the stability of the hyperbolic cross discrete Fourier transform , 2011, Numerische Mathematik.

[12]  Armin Iske,et al.  Curvature analysis of frequency modulated manifolds in dimensionality reduction , 2011 .

[13]  Wolfgang Dahmen,et al.  Fast high-dimensional approximation with sparse occupancy trees , 2011, J. Comput. Appl. Math..

[14]  Daniela Rosca,et al.  A New Hybrid Method for Image Approximation Using the Easy Path Wavelet Transform , 2011, IEEE Transactions on Image Processing.

[15]  Steffen Dereich,et al.  Multilevel Monte Carlo algorithms for L\'{e}vy-driven SDEs with Gaussian correction , 2011, 1101.1369.

[16]  Wolfgang Dahmen,et al.  Convergence Rates for Greedy Algorithms in Reduced Basis Methods , 2010, SIAM J. Math. Anal..

[17]  Wang-Q Lim,et al.  Compactly supported shearlets are optimally sparse , 2010, J. Approx. Theory.

[18]  Frank Wübbeling,et al.  Inverse und schlecht gestellte Probleme , 2011 .

[19]  Christian Bender,et al.  Dual pricing of multi-exercise options under volume constraints , 2011, Finance Stochastics.

[20]  Stephan Dahlke,et al.  Adaptive wavelet methods and sparsity reconstruction for inverse heat conduction problems , 2010, Adv. Comput. Math..

[21]  Tobias Jahnke,et al.  Solving chemical master equations by adaptive wavelet compression , 2010, J. Comput. Phys..

[22]  Dajana Conte,et al.  Mathematical Modelling and Numerical Analysis an Error Analysis of the Multi-configuration Time-dependent Hartree Method of Quantum Dynamics , 2022 .

[23]  Fred J. Hickernell,et al.  Multi-level Monte Carlo algorithms for infinite-dimensional integration on RN , 2010, J. Complex..

[24]  G. Kutyniok,et al.  Construction of Compactly Supported Shearlet Frames , 2010, 1003.5481.

[25]  G. Kutyniok,et al.  Irregular Shearlet Frames: Geometry and Approximation Properties , 2010, 1002.2657.

[26]  G. Teschke,et al.  Accelerated projected steepest descent method for nonlinear inverse problems with sparsity constraints , 2010 .

[27]  Daniel Rudolf,et al.  Error bounds for computing the expectation by Markov chain Monte Carlo , 2009, Monte Carlo Methods Appl..

[28]  Michael Döhler,et al.  Nonequispaced Hyperbolic Cross Fast Fourier Transform , 2010, SIAM J. Numer. Anal..

[29]  Winfried Sickel,et al.  Best m-term aproximation and tensor product of Sobolev and Besov spaces-the case of non-compact embeddings , 2010 .

[30]  Christian Bender,et al.  Error Criteria for Numerical Solutions of Backward SDEs , 2010 .

[31]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[32]  Lars Grasedyck,et al.  Polynomial Approximation in Hierarchical Tucker Format by Vector – Tensorization , 2010 .

[33]  R. Schilling,et al.  Weak Order for the Discretization of the Stochastic Heat Equation Driven by Impulsive Noise , 2009, 0911.4681.

[34]  W. Hackbusch,et al.  Black Box Low Tensor-Rank Approximation Using Fiber-Crosses , 2009 .

[35]  E. Novak,et al.  Optimal Order of Convergence and (In)Tractability of Multivariate Approximation of Smooth Functions , 2009 .

[36]  Henryk Wozniakowski,et al.  Approximation of infinitely differentiable multivariate functions is intractable , 2009, J. Complex..

[37]  K. Bredies,et al.  Regularization with non-convex separable constraints , 2009 .

[38]  Erwan Faou,et al.  Computing Semiclassical Quantum Dynamics with Hagedorn Wavepackets , 2009, SIAM J. Sci. Comput..

[39]  D. Lorenz,et al.  Elastic-net regularization: error estimates and active set methods , 2009, 0905.0796.

[40]  G. Plonka.  The Easy Path Wavelet Transform: A New Adaptive Wavelet Transform for Sparse Representation of Two-Dimensional Data , 2009 .

[41]  D. Lorenz,et al.  Greedy solution of ill-posed problems: error bounds and exact inversion , 2009, 0904.0154.

[42]  Dirk A. Lorenz,et al.  A generalized conditional gradient method and its connection to an iterative shrinkage method , 2009, Comput. Optim. Appl..

[43]  Yonina C. Eldar.  Compressed Sensing of Analog Signals in Shift-Invariant Spaces , 2008, IEEE Transactions on Signal Processing.

[44]  Yonina C. Eldar,et al.  Blind Multiband Signal Reconstruction: Compressed Sensing for Analog Signals , 2007, IEEE Transactions on Signal Processing.

[45]  K. Ritter,et al.  Variable Subspace Sampling and Multi-level Algorithms , 2009 .

[46]  M. Fornasier,et al.  Multilevel Preconditioning for Adaptive Sparse Optimization , 2009 .

[47]  Jianwei Ma,et al.  A Review of Curvelets and Recent Applications , 2009 .

[48]  G. Teschke,et al.  A compressive Landweber iteration for solving ill-posed inverse problems , 2008 .

[49]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[50]  Dirk A. Lorenz,et al.  Iterated Hard Shrinkage for Minimization Problems with Sparsity Constraints , 2008, SIAM J. Sci. Comput..

[51]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[52]  I. Daubechies,et al.  Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints , 2007, 0706.4297.

[53]  Pierre Vandergheynst,et al.  Compressed Sensing and Redundant Dictionaries , 2007, IEEE Transactions on Information Theory.

[54]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[55]  Gerlind Plonka-Hoch,et al.  The Easy Path Wavelet Transform: A New Adaptive Wavelet Transform for Sparse Representation of Two-Dimensional Data , 2008, Multiscale Model. Simul..

[56]  I. Daubechies,et al.  On Some Iterative Concepts for Image Restoration , 2008 .

[57]  M. Fornasier.  Domain decomposition methods for linear inverse problems with sparsity constraints , 2007 .

[58]  Dirk A. Lorenz,et al.  A generalized conditional gradient method for nonlinear operator equations with sparsity constraints , 2007 .

[59]  Richard G. Baraniuk,et al.  Theory and Implementation of an Analog-to-Information Converter using Random Demodulation , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[60]  G. Teschke.  Multi-frame representations in linear inverse problems with mixed multi-constraints , 2007 .

[61]  I. Daubechies,et al.  Iteratively solving linear inverse problems under general convex constraints , 2007 .

[62]  Ronny Ramlau,et al.  A Tikhonov-based projection iteration for nonlinear Ill-posed problems with sparsity constraints , 2006, Numerische Mathematik.

[63]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[64]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[65]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[66]  E.J. Candes.  Compressive Sampling , 2022 .

[67]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[68]  G. Aubert,et al.  Modeling Very Oscillating Signals. Application to Image Processing , 2005 .

[69]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[70]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[71]  E. Novak,et al.  Optimal approximation of elliptic problems by linear and nonlinear mappings I , 2006, J. Complex..

[72]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[73]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[74]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[75]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[76]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[77]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[78]  Martin Greiner,et al.  Wavelets , 2018, Complex..

[79]  C. W. Groetsch,et al.  Inverse Problems in the Mathematical Sciences , 1993 .

[80]  V. A. Morozov,et al.  Methods for Solving Incorrectly Posed Problems , 1984 .

[81]  C. W. Groetsch,et al.  The theory of Tikhonov regularization for Fredholm equations of the first kind , 1984 .

[82]  H. Yserentant,et al.  On the Complexity of the Electronic Schrödinger Equation , 2022 .

[83]  E. Ullmann,et al.  Preconditioning Stochastic Galerkin Saddle Point Problems , 2022 .