Atomic Norm Denoising With Applications to Line Spectral Estimation

Motivated by recent work on atomic norms in inverse problems, we propose a new approach to line spectral estimation that provides theoretical guarantees for the mean-squared-error (MSE) performance in the presence of noise and without knowledge of the model order. We propose an abstract theory of denoising with atomic norms and specialize this theory to provide a convex optimization problem for estimating the frequencies and phases of a mixture of complex exponentials. We show that the associated convex optimization problem can be solved in polynomial time via semidefinite programming (SDP). We also show that the SDP can be approximated by an l1-regularized least-squares problem that achieves nearly the same error rate as the SDP but can scale to much larger problems. We compare both SDP and l1-based approaches with classical line spectral analysis methods and demonstrate that the SDP outperforms the l1 optimization which outperforms MUSIC, Cadzow's, and Matrix Pencil approaches in terms of MSE over a wide range of signal-to-noise ratios.

[1]  A. Schaeffer Inequalities of A. Markoff and S. Bernstein for polynomials and related functions , 1941 .

[2]  U. Grenander,et al.  Toeplitz Forms And Their Applications , 1958 .

[3]  H. Akaike Fitting autoregressive models for prediction , 1969 .

[4]  A. Atzmon A moment problem for positive measures on the unit disc. , 1975 .

[5]  H. Robbins,et al.  Maximally dependent random variables. , 1976, Proceedings of the National Academy of Sciences of the United States of America.

[6]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[7]  S.M. Kay,et al.  Spectrum analysis—A modern perspective , 1981, Proceedings of the IEEE.

[8]  R. Kumaresan,et al.  Estimation of frequencies of multiple sinusoids: Making linear prediction perform like maximum likelihood , 1982, Proceedings of the IEEE.

[9]  Kai-Bor Yu,et al.  Total least squares approach for frequency estimation using linear prediction , 1987, IEEE Trans. Acoust. Speech Signal Process..

[10]  James A. Cadzow,et al.  Signal enhancement-a composite property mapping algorithm , 1988, IEEE Trans. Acoust. Speech Signal Process..

[11]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[12]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[13]  Tapan K. Sarkar,et al.  Matrix pencil method for estimating parameters of exponentially damped/undamped sinusoids in noise , 1990, IEEE Trans. Acoust. Speech Signal Process..

[14]  Randolph L. Moses,et al.  High resolution radar target modeling using a modified Prony estimator , 1992 .

[15]  R. Vautard,et al.  Singular-spectrum analysis: a toolkit for short, noisy chaotic signals , 1992 .

[16]  M. R. Osborne,et al.  On the consistency of Prony's method and related algorithms , 1992 .

[17]  Petre Stoica List of references on spectral line analysis , 1993, Signal Process..

[18]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[19]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[20]  J. A. Stewart,et al.  Nonlinear Time Series Analysis , 2015 .

[21]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[22]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[23]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[24]  R. Curto An operator-theoretic approach to truncated moment problems , 1997 .

[25]  Piero Barone,et al.  Prony methods in NMR spectroscopy , 1997, Int. J. Imaging Syst. Technol..

[26]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[27]  David L. Donoho,et al.  Application of basis pursuit in spectrum estimation , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[28]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[29]  Imaging and time reversal in random media , 2001 .

[30]  Kim-Chuan Toh,et al.  SDPT3 — a Matlab software package for semidefinite-quadratic-linear programming, version 3.0 , 2001 .

[31]  S. Szarek,et al.  Chapter 8 - Local Operator Theory, Random Matrices and Banach Spaces , 2001 .

[32]  Andrew L. Rukhin,et al.  Analysis of Time Series Structure SSA and Related Techniques , 2002, Technometrics.

[33]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[34]  R. Plemmons,et al.  Structured low rank approximation , 2003 .

[35]  Martin Vetterli,et al.  Low-sampling rate UWB channel characterization and synchronization , 2003, Journal of Communications and Networks.

[36]  A. Megretski Positivity of trigonometric polynomials , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[37]  Tadeusz Lobos,et al.  Advanced spectrum estimation methods for signal analysis in power electronics , 2003, IEEE Trans. Ind. Electron..

[38]  Y. Ritov,et al.  Persistence in high-dimensional linear predictor selection and the virtue of overparametrization , 2004 .

[39]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[40]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[41]  Petre Stoica,et al.  Spectral Analysis of Signals , 2009 .

[42]  H. Carfantan,et al.  A Sparsity-Based Method for the Estimation of Spectral Lines From Irregularly Sampled Data , 2007, IEEE Journal of Selected Topics in Signal Processing.

[43]  B. Dumitrescu Positive Trigonometric Polynomials and Signal Processing Applications , 2007 .

[44]  M. Vetterli,et al.  Sparse Sampling of Signal Innovations , 2008, IEEE Signal Processing Magazine.

[45]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[46]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[47]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[48]  Daniel Potts,et al.  Parameter estimation for exponential sums by approximate Prony method , 2010, Signal Process..

[49]  A. Zhigljavsky Singular Spectrum Analysis for time series: Introduction to this special issue , 2010 .

[50]  A. Robert Calderbank,et al.  Sensitivity to Basis Mismatch in Compressed Sensing , 2011, IEEE Trans. Signal Process..

[51]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[52]  Benjamin Recht,et al.  Atomic norm denoising with applications to line spectral estimation , 2011, Allerton.

[53]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[54]  X. Andrade,et al.  Application of compressed sensing to the simulation of atomic systems , 2012, Proceedings of the National Academy of Sciences.

[55]  A. Álvarez,et al.  On the trigonometric moment problem , 2011, Ergodic Theory and Dynamical Systems.

[56]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[57]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[58]  Marco F. Duarte,et al.  Spectral compressive sensing , 2013 .

[59]  Badri Narayan Bhaskar,et al.  Compressed Sensing o the Grid , 2013 .

[60]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.