Compressive Sampling Using Annihilating Filter-Based Low-Rank Interpolation

While the recent theory of compressed sensing provides an opportunity to overcome the Nyquist limit in recovering sparse signals, a solution approach usually takes the form of an inverse problem of an unknown signal, which is crucially dependent on specific signal representation. In this paper, we propose a drastically different two-step Fourier compressive sampling framework in a continuous domain that can be implemented via measurement domain interpolation, after which signal reconstruction can be done using classical analytic reconstruction methods. The main idea originates from the fundamental duality between the sparsity in the primary space and the low-rankness of a structured matrix in the spectral domain, showing that a low-rank interpolator in the spectral domain can enjoy all of the benefits of sparse recovery with performance guarantees. Most notably, the proposed low-rank interpolation approach can be regarded as a generalization of recent spectral compressed sensing to recover large classes of finite rate of innovations (FRI) signals at a near-optimal sampling rate. Moreover, for the case of cardinal representation, we can show that the proposed low-rank interpolation scheme will benefit from inherent regularization and an optimal incoherence parameter. Using a powerful dual certificate and the golfing scheme, we show that the new framework still achieves a near-optimal sampling rate for a general class of FRI signal recovery, while the sampling rate can be further reduced for a class of cardinal splines. Numerical results using various types of FRI signals confirm that the proposed low-rank interpolation approach offers significantly better phase transitions than conventional compressive sampling approaches.

[1]  E.J. Candes Compressive Sampling , 2022 .

[2]  S. Elaydi An introduction to difference equations , 1995 .

[3]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.

[4]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[5]  Martin Vetterli,et al.  Sampling and reconstruction of signals with finite rate of innovation in the presence of noise , 2005, IEEE Transactions on Signal Processing.

[6]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[7]  Michael Unser,et al.  A unified formulation of Gaussian vs. sparse stochastic processes - Part II: Discrete-domain theory , 2011, ArXiv.

[8]  Walter Gautschi,et al.  Norm estimates for inverses of Vandermonde matrices , 1974 .

[9]  Yudong Chen,et al.  Incoherence-Optimal Matrix Completion , 2013, IEEE Transactions on Information Theory.

[10]  Jong Chul Ye,et al.  Sparse + Low Rank Decomposition of Annihilating Filter-based Hankel Matrix for Impulse Noise Removal , 2015, ArXiv.

[11]  Yuxin Chen,et al.  Robust Spectral Compressed Sensing via Structured Matrix Completion , 2013, IEEE Transactions on Information Theory.

[12]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[13]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[14]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[15]  Walter Gautschi,et al.  On inverses of Vandermonde and confluent Vandermonde matrices. II , 1963 .

[16]  Suliana Manley,et al.  Fast live cell imaging at nanometer scale using annihilating filter-based low-rank Hankel matrix approach , 2015, SPIE Optical Engineering + Applications.

[17]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[18]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[19]  Michael Unser,et al.  A general sampling theory for nonideal acquisition devices , 1994, IEEE Trans. Signal Process..

[20]  J. C. Ye,et al.  Acceleration of MR parameter mapping using annihilating filter‐based low rank hankel matrix (ALOHA) , 2016, Magnetic resonance in medicine.

[21]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[22]  Volkan Cevher,et al.  An SVD-free Approach to a Class of Structured Low Rank Matrix Optimization Problems with Application to System Identification , 2013, CDC 2013.

[23]  Ankur Moitra,et al.  Super-resolution, Extremal Functions and the Condition Number of Vandermonde Matrices , 2014, STOC.

[24]  Akram Aldroubi,et al.  B-spline signal processing. II. Efficiency design and applications , 1993, IEEE Trans. Signal Process..

[25]  Jong Chul Ye,et al.  A General Framework for Compressed Sensing and Parallel MRI Using Annihilating Filter Based Low-Rank Hankel Matrix , 2015, IEEE Transactions on Computational Imaging.

[26]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[27]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[28]  Michael Unser,et al.  A unified formulation of Gaussian vs. sparse stochastic processes - Part I: Continuous-domain theory , 2011, ArXiv.

[29]  Thierry Blu,et al.  Extrapolation and Interpolation) , 2022 .

[30]  Jong Chul Ye,et al.  MRI artifact correction using sparse + low‐rank decomposition of annihilating filter‐based hankel matrix , 2017, Magnetic resonance in medicine.

[31]  Jong Chul Ye,et al.  Annihilating Filter-Based Low-Rank Hankel Matrix Approach for Image Inpainting , 2015, IEEE Transactions on Image Processing.

[32]  Dmitry Batenkov,et al.  Numerical stability bounds for algebraic systems of Prony type and their accurate solution by decimation , 2014 .

[33]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[34]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[35]  Emmanuel J. Candès,et al.  Super-Resolution from Noisy Data , 2012, Journal of Fourier Analysis and Applications.

[36]  Gongguo Tang,et al.  Near minimax line spectral estimation , 2013, 2013 47th Annual Conference on Information Sciences and Systems (CISS).

[37]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[38]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[39]  Numerische MathemaUk,et al.  On Inverses of Vandermonde and Confluent Vandermonde Matrices III , 1978 .

[40]  T. Sarkar,et al.  Using the matrix pencil method to estimate the parameters of a sum of complex exponentials , 1995 .

[41]  Dmitry Batenkov,et al.  On the Accuracy of Solving Confluent Prony Systems , 2011, SIAM J. Appl. Math..

[42]  Daniel Boley,et al.  Vandermonde Factorization of a Hankel Matrix ? , 2006 .

[43]  Walter Galitschi,et al.  On inverses of Vandermonde and confluent Vandermonde matrices , 1962 .

[44]  Michael Unser,et al.  B-spline signal processing. I. Theory , 1993, IEEE Trans. Signal Process..

[45]  Helmut Bölcskei,et al.  Deterministic performance analysis of subspace methods for cisoid parameter estimation , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[46]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[47]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[48]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[49]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[50]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[51]  Roland Badeau,et al.  High-resolution spectral analysis of mixtures of complex exponentials modulated by polynomials , 2006, IEEE Transactions on Signal Processing.

[52]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[53]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[54]  G. Rota,et al.  Finite operator calculus , 1975 .

[55]  Tapan K. Sarkar,et al.  Matrix pencil method for estimating parameters of exponentially damped/undamped sinusoids in noise , 1990, IEEE Trans. Acoust. Speech Signal Process..

[56]  Marco F. Duarte,et al.  Spectral compressive sensing , 2013 .

[57]  Jong Chul Ye,et al.  Reference‐free single‐pass EPI Nyquist ghost correction using annihilating filter‐based low rank Hankel matrix (ALOHA) , 2016, Magnetic resonance in medicine.

[58]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[59]  Dmitry Batenkov,et al.  Decimated generalized Prony systems , 2013, 1308.0753.

[60]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[61]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[62]  Michael Unser,et al.  An Introduction to Sparse Stochastic Processes , 2014 .

[63]  Nathan Srebro,et al.  Learning with matrix factorizations , 2004 .

[64]  M. Unser Sampling-50 years after Shannon , 2000, Proceedings of the IEEE.

[65]  P. Lauterbur,et al.  Principles of magnetic resonance imaging : a signal processing perspective , 1999 .