Harnessing Sparsity Over the Continuum: Atomic norm minimization for superresolution

At the core of many sensing and imaging applications, the signal of interest can be modeled as a linear superposition of translated or modulated versions of some template [e.g., a point spread function (PSF) or a Green's function] and the fundamental problem is to estimate the translation or modulation parameters (e.g., delays, locations, or Dopplers) from noisy measurements. This problem is centrally important to not only target localization in radar and sonar, channel estimation in wireless communications, and direction-of-arrival estimation in array signal processing, but also modern imaging modalities such as superresolution single-molecule fluorescence microscopy, nuclear magnetic resonance imaging, and spike localization in neural recordings, among others.

[1]  Wei Dai,et al.  A Tight Converse to the Spectral Resolution Limit via Convex Programming , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[2]  Yuejie Chi,et al.  On the Stable Resolution Limit of Total Variation Regularization for Spike Deconvolution , 2020, IEEE Transactions on Information Theory.

[3]  Nicolas Boumal,et al.  The non-convex Burer-Monteiro approach works on smooth semidefinite programs , 2016, NIPS.

[4]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[5]  Rémi Gribonval,et al.  Recipes for stable linear embeddings from Hilbert spaces to R^m , 2015 .

[6]  Arye Nehorai,et al.  Joint Sparse Recovery Method for Compressed Sensing With Structured Dictionary Mismatches , 2013, IEEE Transactions on Signal Processing.

[7]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[8]  Jean-Jacques Fuchs Extension of the Pisarenko method to sparse linear arrays , 1997, IEEE Trans. Signal Process..

[9]  Gongguo Tang,et al.  Approximate support recovery of atomic line spectral estimation: A tale of resolution and precision , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[10]  J. Goodenough,et al.  Scientific Background on the Nobel Prize in Chemistry 2019-Lithium-ion batteries , 2010 .

[11]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[12]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[13]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[14]  Ben Adcock,et al.  Generalized Sampling and Infinite-Dimensional Compressed Sensing , 2015, Foundations of Computational Mathematics.

[15]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[16]  V. Pisarenko The Retrieval of Harmonics from a Covariance Function , 1973 .

[17]  Yonina C. Eldar,et al.  Strong Duality of Sparse Functional Optimization , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Tryphon T. Georgiou The Carathéodory–Fejér–Pisarenko Decomposition and Its Multivariable Counterpart , 2007, IEEE Transactions on Automatic Control.

[19]  Renato D. C. Monteiro,et al.  A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization , 2003, Math. Program..

[20]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[21]  Reinhard Heckel,et al.  Generalized Line Spectral Estimation via Convex Optimization , 2016, IEEE Transactions on Information Theory.

[22]  Gabriel Peyré,et al.  Exact Support Recovery for Sparse Spikes Deconvolution , 2013, Foundations of Computational Mathematics.

[23]  Claire Boyer,et al.  Adapting to unknown noise level in sparse deconvolution , 2016, ArXiv.

[24]  B. Dumitrescu Positive Trigonometric Polynomials and Signal Processing Applications , 2007 .

[25]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.

[26]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[27]  Hemant Tyagi,et al.  Sparse non-negative super-resolution - simplified and stabilised , 2018, Applied and Computational Harmonic Analysis.

[28]  Carlos Fernandez-Granda,et al.  Super-resolution of point sources via convex programming , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[29]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[30]  Yudong Chen,et al.  Harnessing Structures in Big Data via Guaranteed Low-Rank Matrix Estimation: Recent Theory and Fast Algorithms via Convex and Nonconvex Optimization , 2018, IEEE Signal Processing Magazine.

[31]  Ankur Moitra,et al.  Super-resolution, Extremal Functions and the Condition Number of Vandermonde Matrices , 2014, STOC.

[32]  Di Guo,et al.  Hankel Matrix Nuclear Norm Regularized Tensor Completion for $N$-dimensional Exponential Signals , 2016, IEEE Transactions on Signal Processing.

[33]  Benjamin Recht,et al.  The alternating descent conditional gradient method for sparse inverse problems , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[34]  Mark Bates,et al.  Super-resolution fluorescence microscopy. , 2009, Annual review of biochemistry.

[35]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[36]  G. Peyré,et al.  Sparse spikes super-resolution on thin grids II: the continuous basis pursuit , 2017 .

[37]  Yuxin Chen,et al.  Robust Spectral Compressed Sensing via Structured Matrix Completion , 2013, IEEE Transactions on Information Theory.

[38]  Ben Adcock,et al.  BREAKING THE COHERENCE BARRIER: A NEW THEORY FOR COMPRESSED SENSING , 2013, Forum of Mathematics, Sigma.

[39]  Jong Chul Ye,et al.  A General Framework for Compressed Sensing and Parallel MRI Using Annihilating Filter Based Low-Rank Hankel Matrix , 2015, IEEE Transactions on Computational Imaging.

[40]  AdcockBen,et al.  Generalized Sampling and Infinite-Dimensional Compressed Sensing , 2016 .

[41]  Gongguo Tang,et al.  Sparse recovery over continuous dictionaries-just discretize , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[42]  M. Vetterli,et al.  Sparse Sampling of Signal Innovations , 2008, IEEE Signal Processing Magazine.

[43]  A. Robert Calderbank,et al.  Sensitivity to Basis Mismatch in Compressed Sensing , 2011, IEEE Trans. Signal Process..

[44]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[45]  Petre Stoica,et al.  Maximum likelihood estimation of the parameters of multiple sinusoids from noisy measurements , 1989, IEEE Trans. Acoust. Speech Signal Process..

[46]  Lei Zhu,et al.  Faster STORM using compressed sensing , 2012, Nature Methods.

[47]  Carlos Fernandez-Granda Support detection in super-resolution , 2013, ArXiv.

[48]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[49]  C. Carathéodory Über den Variabilitätsbereich der Koeffizienten von Potenzreihen, die gegebene Werte nicht annehmen , 1907 .

[50]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[51]  Yuejie Chi,et al.  Off-the-Grid Line Spectrum Denoising and Estimation With Multiple Measurement Vectors , 2014, IEEE Transactions on Signal Processing.

[52]  Yuejie Chi,et al.  Super-Resolution Image Reconstruction for High-Density Three-Dimensional Single-Molecule Microscopy , 2017, IEEE Transactions on Computational Imaging.

[53]  B. Reznick,et al.  Sums of squares of real polynomials , 1995 .

[54]  Upamanyu Madhow,et al.  Newtonized Orthogonal Matching Pursuit: Frequency Estimation Over the Continuum , 2015, IEEE Transactions on Signal Processing.

[55]  C. Carathéodory Über den variabilitätsbereich der fourier’schen konstanten von positiven harmonischen funktionen , 1911 .

[56]  Yuejie Chi,et al.  Guaranteed Blind Sparse Spikes Deconvolution via Lifting and Convex Optimization , 2015, IEEE Journal of Selected Topics in Signal Processing.

[57]  Yuejie Chi,et al.  Quantized Spectral Compressed Sensing: Cramer–Rao Bounds and Recovery Algorithms , 2017, IEEE Transactions on Signal Processing.

[58]  Yuxin Chen,et al.  Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview , 2018, IEEE Transactions on Signal Processing.

[59]  Jean-Jacques Fuchs,et al.  Sparsity and uniqueness for some specific under-determined linear systems , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[60]  Gongguo Tang,et al.  Near minimax line spectral estimation , 2013, 2013 47th Annual Conference on Information Sciences and Systems (CISS).

[61]  Scientific Background on the Nobel Prize in Chemistry 2011 The DiSCovery of QuaSiCrySTalS , 2011 .

[62]  Gongguo Tang,et al.  Atomic Norm Denoising With Applications to Line Spectral Estimation , 2012, IEEE Transactions on Signal Processing.

[63]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[64]  J. Lippincott-Schwartz,et al.  Imaging Intracellular Fluorescent Proteins at Nanometer Resolution , 2006, Science.

[65]  Mathews Jacob,et al.  Convex Recovery of Continuous Domain Piecewise Constant Images From Nonuniform Fourier Samples , 2017, IEEE Transactions on Signal Processing.

[66]  Yonina C. Eldar,et al.  Functional Nonlinear Sparse Models , 2020, IEEE Transactions on Signal Processing.

[67]  G. Golub,et al.  Eigenvalue computation in the 20th century , 2000 .

[68]  Benjamin Recht,et al.  Superresolution without separation , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[69]  P. P. Vaidyanathan,et al.  On the Role of the Bounded Lemma in the SDP Formulation of Atomic Norm Problems , 2017, IEEE Signal Processing Letters.

[70]  Gabriel Peyré,et al.  Support Recovery for Sparse Super-Resolution of Positive Measures , 2017 .

[71]  Max Born,et al.  Principles of optics - electromagnetic theory of propagation, interference and diffraction of light (7. ed.) , 1999 .

[72]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[73]  Weiyu Xu,et al.  Robust recovery of complex exponential signals from random Gaussian projections via low rank Hankel matrix reconstruction , 2015, Applied and computational harmonic analysis.

[74]  G. Peyré,et al.  Sparse regularization on thin grids I: the Lasso , 2017 .

[75]  L. Scharf,et al.  A Prony method for noisy data: Choosing the signal components and selecting the order in exponential signal models , 1984, Proceedings of the IEEE.

[76]  Emmanuel J. Candès,et al.  Super-Resolution of Positive Sources: The Discrete Setup , 2015, SIAM J. Imaging Sci..

[77]  Xiaodong Wang,et al.  Millimeter-Wave Beamformed Full-Dimensional MIMO Channel Estimation Based on Atomic Norm Minimization , 2017, IEEE Transactions on Communications.

[78]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[79]  Yuejie Chi,et al.  Stable Separation and Super-Resolution of Mixture Models , 2015, Applied and Computational Harmonic Analysis.

[80]  J. Lasserre Moments, Positive Polynomials And Their Applications , 2009 .

[81]  Michael J Rust,et al.  Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM) , 2006, Nature Methods.

[82]  Louis L. Scharf,et al.  Two-dimensional modal analysis based on maximum likelihood , 1994, IEEE Trans. Signal Process..

[83]  Zhihui Zhu,et al.  Super-Resolution of Complex Exponentials From Modulations With Unknown Waveforms , 2016, IEEE Transactions on Information Theory.

[84]  S.M. Kay,et al.  Spectrum analysis—A modern perspective , 1981, Proceedings of the IEEE.

[85]  Zhihui Zhu,et al.  Super-resolution in SAR imaging: Analysis with the atomic norm , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[86]  Yuejie Chi Convex relaxations of spectral sparsity for robust super-resolution and line spectrum estimation , 2017, Optical Engineering + Applications.

[87]  Gongguo Tang,et al.  Demixing Sines and Spikes: Robust Spectral Super-resolution in the Presence of Outliers , 2016, ArXiv.

[88]  Xiaodong Li,et al.  Compressed Sensing and Matrix Completion with Constant Proportion of Corruptions , 2011, Constructive Approximation.

[89]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[90]  Yuxin Chen,et al.  Compressive Two-Dimensional Harmonic Retrieval via Atomic Norm Minimization , 2015, IEEE Transactions on Signal Processing.

[91]  Michael D. Mason,et al.  Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. , 2006, Biophysical journal.

[92]  Bhaskar D. Rao,et al.  Performance analysis of Root-Music , 1989, IEEE Trans. Acoust. Speech Signal Process..

[93]  Georgios B. Giannakis,et al.  Compressed Sensing for Wideband Cognitive Radios , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[94]  D. Donoho,et al.  Minimax risk of matrix denoising by singular value thresholding , 2013, 1304.2085.

[95]  Weiyu Xu,et al.  Precise semidefinite programming formulation of atomic norm minimization for recovering d-dimensional (D ≥ 2) off-the-grid frequencies , 2013, 2014 Information Theory and Applications Workshop (ITA).

[96]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[97]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[98]  Stephen J. Wright,et al.  Forward–Backward Greedy Algorithms for Atomic Norm Regularization , 2014, IEEE Transactions on Signal Processing.

[99]  Petre Stoica,et al.  Introduction to spectral analysis , 1997 .

[100]  Mark Bates,et al.  Three-Dimensional Super-Resolution Imaging by Stochastic Optical Reconstruction Microscopy , 2008, Science.

[101]  Hassan Mansour,et al.  An Optimization View of MUSIC and Its Extension to Missing Data , 2018, ArXiv.

[102]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[103]  Olav Tirkkonen,et al.  MmWave channel estimation via atomic norm minimization for multi-user hybrid precoding , 2018, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[104]  Tapan K. Sarkar,et al.  Matrix pencil method for estimating parameters of exponentially damped/undamped sinusoids in noise , 1990, IEEE Trans. Acoust. Speech Signal Process..

[105]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.