Computing a k-sparse n-length Discrete Fourier Transform using at most 4k samples and O(k log k) complexity

Given an n-length input signal x, it is well known that its Discrete Fourier Transform (DFT), X, can be computed in O(nlogn) complexity using a Fast Fourier Transform. If the spectrum X is exactly k-sparse (where k <;<; n), can we do better? We show that asymptotically in k and n, when k is sub-linear in n (i.e., k ∝ nδ where 0 <; δ <; 1), and the support of the non-zero DFT coefficients is uniformly random, we can exploit this sparsity in two fundamental ways (i) sample complexity: we need only M = rk deterministically chosen samples of the input signal x (where r <; 4 when 0 <; δ <; 0.99); and (ii) computational complexity: we can reliably compute the DFT X using O(k log k) operations, where the constants in the big Oh are small. Our algorithm succeeds with high probability, with the probability of failure vanishing to zero asymptotically in the number of samples acquired, M. Our approach is based on filterless subsampling of the input signal x using a small set of carefully chosen uniform subsampling patterns guided by the Chinese Remainder Theorem (CRT). Specifically, our subsampling operation on x is designed to create aliasing patterns on the spectrum X that "look like" parity-check constraints of good erasure-correcting sparse-graph codes. We show how computing the sparse DFT X is equivalent to decoding of these sparse-graph codes and is low in both sample complexity and decoding complexity. We accordingly dub our algorithm the FFAST (Fast Fourier Aliasing-based Sparse Transform) algorithm. In our analysis, we rigorously connect our CRT based graph constructions to random sparse-graph codes based on a balls-and-bins model and analyze the convergence behavior of the latter using well-studied density evolution techniques from coding theory. We provide simulation results in Section IV that corroborate our theoretical findings, and validate the empirical performance of the FFAST algorithm.

[1]  Kannan Ramchandran,et al.  A robust R-FFAST framework for computing a k-sparse n-length DFT in O(k log n) sample complexity using sparse-graph codes , 2014, 2014 IEEE International Symposium on Information Theory.

[2]  Thierry Blu,et al.  Extrapolation and Interpolation) , 2022 .

[3]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[4]  David P. Woodruff,et al.  Lower bounds for sparse recovery , 2010, SODA '10.

[5]  M. Vetterli,et al.  Sparse Sampling of Signal Innovations , 2008, IEEE Signal Processing Magazine.

[6]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[7]  David P. Woodruff,et al.  (1 + eps)-Approximate Sparse Recovery , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[8]  V. Pisarenko The Retrieval of Harmonics from a Covariance Function , 1973 .

[9]  Daniel A. Spielman,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[10]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[11]  Mark A. Iwen,et al.  Empirical evaluation of a sub-linear time sparse DFT algorithm , 2007 .

[12]  Yonina C. Eldar,et al.  From Theory to Practice: Sub-Nyquist Sampling of Sparse Wideband Analog Signals , 2009, IEEE Journal of Selected Topics in Signal Processing.

[13]  Kannan Ramchandran,et al.  A hybrid DFT-LDPC framework for fast, efficient and robust compressive sensing , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[14]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[15]  Kannan Ramchandran,et al.  PhaseCode: Fast and Efficient Compressive Phase Retrieval Based on Sparse-Graph Codes , 2017, IEEE Transactions on Information Theory.

[16]  Piotr Indyk,et al.  Nearly optimal sparse fourier transform , 2012, STOC '12.

[17]  Eli Upfal,et al.  Probability and Computing: Randomized Algorithms and Probabilistic Analysis , 2005 .

[18]  Martin Vetterli,et al.  A Fast Hadamard Transform for Signals With Sublinear Sparsity in the Transform Domain , 2015, IEEE Trans. Inf. Theory.

[19]  Piotr Indyk,et al.  Simple and practical algorithm for sparse Fourier transform , 2012, SODA.

[20]  M. Luby,et al.  Verification Codes : Simple Low-Density Parity-Check Codes for Large Alphabets , 2002 .

[21]  Sudipto Guha,et al.  Near-optimal sparse fourier representations via sampling , 2002, STOC '02.

[22]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[23]  Michael Luby,et al.  A digital fountain approach to reliable distribution of bulk data , 1998, SIGCOMM '98.

[24]  A.C. Gilbert,et al.  A Tutorial on Fast Fourier Sampling , 2008, IEEE Signal Processing Magazine.

[25]  Piotr Indyk,et al.  Sample-optimal average-case sparse Fourier Transform in two dimensions , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[26]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[27]  Anna C. Gilbert,et al.  Improved time bounds for near-optimal sparse Fourier representations , 2005, SPIE Optics + Photonics.

[28]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[29]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[30]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[31]  Steven Kay,et al.  A Fast and Accurate Single Frequency Estimator , 2022 .

[32]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[33]  Allan O. Steinhardt,et al.  Fast algorithms for digital signal processing , 1986, Proceedings of the IEEE.

[34]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[35]  C. Temperton Self-sorting mixed-radix fast Fourier transforms , 1983 .

[36]  Mark A. Iwen,et al.  Combinatorial Sublinear-Time Fourier Algorithms , 2010, Found. Comput. Math..