Sparse Recovery Beyond Compressed Sensing: Separable Nonlinear Inverse Problems

Extracting information from nonlinear measurements is a fundamental challenge in data analysis. In this work, we consider separable inverse problems, where the data are modeled as a linear combination of functions that depend nonlinearly on certain parameters of interest. These parameters may represent neuronal activity in a human brain, frequencies of electromagnetic waves, fluorescent probes in a cell, or magnetic relaxation times of biological tissues. Separable nonlinear inverse problems can be reformulated as underdetermined sparse-recovery problems, and solved using convex programming. This approach has had empirical success in a variety of domains, from geophysics to medical imaging, but lacks a theoretical justification. In particular, compressed-sensing theory does not apply, because the measurement operators are deterministic and violate incoherence conditions such as the restricted-isometry property. Our main contribution is a theory for sparse recovery adapted to deterministic settings. We show that convex programming succeeds in recovering the parameters of interest, as long as their values are sufficiently distinct with respect to the correlation structure of the measurement operator. The theoretical results are illustrated through numerical experiments for two applications: heat-source localization and estimation of brain activity from electroencephalography data.

[1]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[2]  Hemant Tyagi,et al.  Sparse non-negative super-resolution - simplified and stabilised , 2018, Applied and Computational Harmonic Analysis.

[3]  Liliana Borcea,et al.  Resolution Analysis of Imaging with ℓ1 Optimization , 2015, SIAM J. Imaging Sci..

[4]  Gerald B. Folland,et al.  Real Analysis: Modern Techniques and Their Applications , 1984 .

[5]  Shai Dekel,et al.  Robust Recovery of Stream of Pulses using Convex Optimization , 2014, ArXiv.

[6]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[7]  Mahmood R. Azimi-Sadjadi,et al.  Localization of near-field sources in sonar data using the sparse representation framework , 2011, OCEANS'11 MTS/IEEE KONA.

[8]  Marco A. López,et al.  Semi-infinite programming , 2007, Eur. J. Oper. Res..

[9]  D. Louis Collins,et al.  Unbiased average age-appropriate atlases for pediatric studies , 2011, NeuroImage.

[10]  Richard M. Leahy,et al.  Electromagnetic brain mapping , 2001, IEEE Signal Process. Mag..

[11]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[12]  P. Riel,et al.  Lp-NORM DECONVOLUTION1 , 1990 .

[13]  Andrew Thompson,et al.  A Bridge Between Past and Present: Exchange and Conditional Gradient Methods are Equivalent , 2018 .

[14]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[15]  Benjamin Recht,et al.  DeepLoco: Fast 3D Localization Microscopy Using Neural Networks , 2018, bioRxiv.

[16]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[17]  J. Claerbout,et al.  Robust Modeling With Erratic Data , 1973 .

[18]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[19]  Théodore Papadopoulo,et al.  OpenMEEG: opensource software for quasistatic bioelectromagnetics , 2010, Biomedical engineering online.

[20]  P. Riel,et al.  Lp-Norm Deconvolution , 1989 .

[21]  Pier Luigi Dragotti,et al.  A Sampling Framework for Solving Physics-Driven Inverse Source Problems , 2017, IEEE Transactions on Signal Processing.

[22]  Liliana Borcea,et al.  Resolution analysis of imaging with $\ell_1$ optimization , 2015 .

[23]  Emmanuel Soubies,et al.  The sliding Frank–Wolfe algorithm and its application to super-resolution microscopy , 2018, Inverse Problems.

[24]  Kenneth O. Kortanek,et al.  Semi-Infinite Programming: Theory, Methods, and Applications , 1993, SIAM Rev..

[25]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[26]  Martin Burger,et al.  Locally Sparse Reconstruction Using the $\ell^{1,\infty}$-Norm , 2014, 1405.5908.

[27]  Benjamin Recht,et al.  The alternating descent conditional gradient method for sparse inverse problems , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[28]  Ernst Fernando Lopes Da Silva Niedermeyer,et al.  Electroencephalography, basic principles, clinical applications, and related fields , 1982 .

[29]  Geert Leus,et al.  Aliasing-Free Wideband Beamforming Using Sparse Signal Representation , 2011, IEEE Transactions on Signal Processing.

[30]  Stanley Osher,et al.  Heat source identification based on l1 constrained minimization , 2014 .

[31]  G. Golub,et al.  Separable nonlinear least squares: the variable projection method and its applications , 2003 .

[32]  Carlos Fernandez-Granda Support detection in super-resolution , 2013, ArXiv.

[33]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[34]  Ismail Sahar,et al.  Super-Resolution , 2014, Encyclopedia of Biometrics.

[35]  Benjamin Recht,et al.  Superresolution without separation , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[36]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[37]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[38]  Dezhong Yao,et al.  Lp Norm Iterative Sparse Solution for EEG Source Localization , 2007, IEEE Transactions on Biomedical Engineering.

[39]  K. Puschmann,et al.  On super-resolution in astronomical imaging , 2005 .

[40]  Thierry Blu,et al.  FRI Sampling With Arbitrary Kernels , 2013, IEEE Transactions on Signal Processing.

[41]  Gabriel Peyré,et al.  Exact Support Recovery for Sparse Spikes Deconvolution , 2013, Foundations of Computational Mathematics.

[42]  Gene H. Golub,et al.  The differentiation of pseudo-inverses and non-linear least squares problems whose variables separate , 1972, Milestones in Matrix Computation.

[43]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[44]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[45]  Michael D. Mason,et al.  Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. , 2006, Biophysical journal.

[46]  H. L. Taylor,et al.  Deconvolution with the l 1 norm , 1979 .

[47]  Parikshit Shah,et al.  Robust line spectral estimation , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[48]  D. Slepian Prolate spheroidal wave functions, fourier analysis, and uncertainty — V: the discrete case , 1978, The Bell System Technical Journal.

[49]  Laurent Daudet,et al.  Compressive Sensing in Acoustic Imaging , 2015 .

[50]  J. Fermaglich Electric Fields of the Brain: The Neurophysics of EEG , 1982 .

[51]  D. Donev Prolate Spheroidal Wave Functions , 2017 .

[52]  G. Peyré,et al.  A numerical exploration of compressed sampling recovery , 2010 .

[53]  Emmanuel J. Candès,et al.  Super-Resolution from Noisy Data , 2012, Journal of Fourier Analysis and Applications.

[54]  S. Levy,et al.  Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution , 1981 .

[55]  D. L. Schomer,et al.  Niedermeyer's Electroencephalography: Basic Principles, Clinical Applications, and Related Fields , 2012 .

[56]  Gabriel Peyr'e,et al.  The geometry of off-the-grid compressed sensing , 2020 .

[57]  J. Maltez,et al.  Evaluation of L1 and L2 minimum norm performances on EEG localizations , 2004, Clinical Neurophysiology.

[58]  Ali Mohammad-Djafari,et al.  Super-Resolution: A Short Review, A New Method Based on Hidden Markov Modeling of HR Image and Future Challenges , 2009, Comput. J..

[59]  Ankur Moitra,et al.  Super-resolution, Extremal Functions and the Condition Number of Vandermonde Matrices , 2014, STOC.

[60]  Gongguo Tang Atomic Decomposition of Mixtures of Translation-Invariant Signals , 2013 .

[61]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[62]  Thierry Blu,et al.  Towards Generalized FRI Sampling With an Application to Source Resolution in Radioastronomy , 2017, IEEE Transactions on Signal Processing.

[63]  W. Rudin Real and complex analysis , 1968 .

[64]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[65]  Elayne Jannette Pegues MATRIX THEORY: BASIC RESULTS AND TECHNIQUES (UNIVERSITEXT) , 2019 .

[66]  Yuxin Chen,et al.  Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution , 2017, Found. Comput. Math..

[67]  Sunli Tang,et al.  Multicompartment magnetic resonance fingerprinting , 2018, Inverse problems.

[68]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[69]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[70]  Petre Stoica,et al.  Spectral Analysis of Signals , 2009 .

[71]  J. Duerk,et al.  Magnetic Resonance Fingerprinting , 2013, Nature.

[72]  J. Lippincott-Schwartz,et al.  Imaging Intracellular Fluorescent Proteins at Nanometer Resolution , 2006, Science.

[73]  W. W. Hansen,et al.  Nuclear Induction , 2011 .

[74]  N. Ricker The Form and Laws of Propagation of Seismic Wavelets , 1951 .

[75]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[76]  Badri Narayan Bhaskar,et al.  Compressed Sensing o the Grid , 2013 .

[77]  Carlos Fernandez-Granda,et al.  A Learning-based Framework for Line-spectra Super-resolution , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[78]  Thierry Blu,et al.  Extrapolation and Interpolation) , 2022 .

[79]  M. Murray,et al.  EEG source imaging , 2004, Clinical Neurophysiology.

[80]  Gongguo Tang,et al.  Approximate support recovery of atomic line spectral estimation: A tale of resolution and precision , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[81]  A. O. Rodríguez,et al.  Principles of magnetic resonance imaging , 2004 .

[82]  F. Santosa,et al.  Linear inversion of ban limit reflection seismograms , 1986 .

[83]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[84]  Yonina C. Eldar,et al.  Phase Retrieval via Matrix Completion , 2011, SIAM Rev..

[85]  C. Comtat,et al.  Fully 4D image reconstruction by estimation of an input function and spectral coefficients , 2007, 2007 IEEE Nuclear Science Symposium Conference Record.

[86]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[87]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[88]  Yen-Hsi Richard Tsai,et al.  Point source identification in nonlinear advection–diffusion–reaction systems , 2012, 1202.2373.

[89]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.

[90]  Roger N Gunn,et al.  Positron Emission Tomography Compartmental Models: A Basis Pursuit Strategy for Kinetic Modeling , 2002, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[91]  Carlos Fernandez-Granda,et al.  Super-resolution of point sources via convex programming , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[92]  Fuzhen Zhang Matrix Theory: Basic Results and Techniques , 1999 .

[93]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[94]  Gabriel Peyré,et al.  A Dual Certificates Analysis of Compressive Off-the-Grid Recovery , 2018, ArXiv.

[95]  Emre Ertin,et al.  Sparsity and Compressed Sensing in Radar Imaging , 2010, Proceedings of the IEEE.

[96]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[97]  Gongguo Tang,et al.  Demixing Sines and Spikes: Robust Spectral Super-resolution in the Presence of Outliers , 2016, ArXiv.

[98]  Konstantin Pieper,et al.  Inverse point source location with the Helmholtz equation on a bounded domain , 2018, Computational Optimization and Applications.

[99]  Carlos Fernandez-Granda,et al.  Deconvolution of Point Sources: A Sampling Theorem and Robustness Guarantees , 2017, Communications on Pure and Applied Mathematics.

[100]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[101]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[102]  F. Harris On the use of windows for harmonic analysis with the discrete Fourier transform , 1978, Proceedings of the IEEE.

[103]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[104]  Pier Luigi Dragotti,et al.  A Universal Sampling Framework for Solving Physics-driven Inverse Source Problems , 2017, ArXiv.

[105]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[106]  Vikas Gulani,et al.  Bayesian estimation of multicomponent relaxation parameters in magnetic resonance fingerprinting , 2018, Magnetic resonance in medicine.