A Bayesian Approach for Online Recovery of Streaming Signals From Compressive Measurements

We consider the progressive reconstruction of a streaming signal from compressive, streaming measurements. We develop a progressive reconstruction algorithm based on sliding window processing, where we reconstruct the streaming signal over small overlapping shifting intervals. Since the consecutive intervals share some common sparse signal vectors, the key idea of this work is to utilize the preliminary information from the preceding interval to improve the performance of the signal recovery algorithm. For this purpose, we propose a novel sparse Bayesian learning (SBL) algorithm, which is highly efficient for recovering streaming signals. One major advantage of SBL is that it provides a measure of uncertainty of the reconstructed signal rather than computing only a point estimate. The proposed SBL algorithm utilizes the previous estimates as well as the correlations among the nonzero coefficients to improve the performance of the algorithm. Since the effect of uncertainty of the reconstructed signal from the preceding interval is specifically taken into account in the recovery process of the current interval, the proposed algorithm is more robust to the error propagation compared to the l1-norm minimization counterpart. Further, we propose a warm-start procedure and derive fast update formulae to reduce the computational cost of the SBL algorithm. In addition, we also discuss the properties of the signal and the underlying approximations which enable the progressive reconstruction of the streaming signal from compressive measurements.

[1]  Vahid Tarokh,et al.  SPARLS: The Sparse RLS Algorithm , 2010, IEEE Transactions on Signal Processing.

[2]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[3]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[4]  E. Candes,et al.  11-magic : Recovery of sparse signals via convex programming , 2005 .

[5]  Namrata Vaswani,et al.  Kalman filtered Compressed Sensing , 2008, 2008 15th IEEE International Conference on Image Processing.

[6]  William F. Moss,et al.  Decay rates for inverses of band matrices , 1984 .

[7]  Kin K. Leung,et al.  Tracking dynamic sparse signals using Hierarchical Bayesian Kalman filters , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  Zhilin Zhang Sparse signal recovery exploiting spatiotemporal correlation , 2012 .

[9]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[10]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[11]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[12]  Ricardo L. De Queiroz,et al.  ON LAPPED TRANSFORMS , 1999 .

[13]  Andrea Montanari,et al.  Graphical Models Concepts in Compressed Sensing , 2010, Compressed Sensing.

[14]  Robert Weigel,et al.  Spread spectrum communications using chirp signals , 2000, IEEE/AFCEA EUROCOMM 2000. Information Systems for Enhanced Public Safety and Security (Cat. No.00EX405).

[15]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[16]  Petros Boufounos,et al.  Compressive sampling for streaming signals with sparse frequency content , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[17]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[18]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[19]  David P. Wipf,et al.  Bayesian methods for finding sparse representations , 2006 .

[20]  Georgios B. Giannakis,et al.  Online Adaptive Estimation of Sparse Signals: Where RLS Meets the $\ell_1$ -Norm , 2010, IEEE Transactions on Signal Processing.

[21]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[22]  Wei Lu,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, 2009 IEEE International Symposium on Information Theory.

[23]  Henrique S. Malvar,et al.  Signal processing with lapped transforms , 1992 .

[24]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[25]  David P. Wipf,et al.  Exploiting the convex-concave penalty for tracking: A novel dynamic reweighted sparse Bayesian learning algorithm , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Namrata Vaswani,et al.  LS-CS-Residual (LS-CS): Compressive Sensing on Least Squares Residual , 2009, IEEE Transactions on Signal Processing.

[27]  P. Bickel,et al.  Approximating the inverse of banded matrices by banded matrices with applications to probability and statistics , 2010, 1002.4545.

[28]  Philip Schniter,et al.  Expectation-Maximization Gaussian-Mixture Approximate Message Passing , 2012, IEEE Transactions on Signal Processing.

[29]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[30]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[31]  José M. F. Moura,et al.  Inversion of block matrices with block banded inverses: application to Kalman-Bucy filtering , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[32]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[33]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[34]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[35]  Michael E. Tipping Bayesian Inference: An Introduction to Principles and Practice in Machine Learning , 2003, Advanced Lectures on Machine Learning.

[36]  Justin Ziniel,et al.  Fast bayesian matching pursuit , 2008, 2008 Information Theory and Applications Workshop.

[37]  Philip Schniter,et al.  Tracking and smoothing of time-varying sparse signals via approximate belief propagation , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[38]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[39]  Justin K. Romberg,et al.  Sparse Recovery of Streaming Signals Using $\ell_1$-Homotopy , 2013, IEEE Transactions on Signal Processing.

[40]  Patrick Flandrin,et al.  Time frequency and chirps , 2001, SPIE Defense + Commercial Sensing.

[41]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[42]  Gilbert Strang Fast transforms: Banded matrices with banded inverses , 2010, Proceedings of the National Academy of Sciences.

[43]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[44]  Alfred O. Hero,et al.  Sparse LMS for system identification , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  Philip Schniter,et al.  Dynamic Compressive Sensing of Time-Varying Signals Via Approximate Message Passing , 2012, IEEE Transactions on Signal Processing.

[46]  José M. F. Moura,et al.  Block matrices with L-block-banded inverse: inversion algorithms , 2005, IEEE Transactions on Signal Processing.

[47]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[48]  Bhaskar D. Rao,et al.  Sparse Signal Recovery With Temporally Correlated Source Vectors Using Sparse Bayesian Learning , 2011, IEEE Journal of Selected Topics in Signal Processing.

[49]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[50]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[51]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[52]  Gérard Meurant,et al.  A Review on the Inverse of Symmetric Tridiagonal and Block Tridiagonal Matrices , 1992, SIAM J. Matrix Anal. Appl..

[53]  Justin K. Romberg,et al.  Dynamic updating for sparse time varying signals , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[54]  Lie Wang,et al.  Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise , 2011, IEEE Transactions on Information Theory.

[55]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[56]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[57]  Marian Codreanu,et al.  Sequential Compressed Sensing With Progressive Signal Reconstruction in Wireless Sensor Networks , 2015, IEEE Transactions on Wireless Communications.

[58]  Wei Lu,et al.  Regularized Modified BPDN for Noisy Sparse Reconstruction With Partial Erroneous Support and Signal Value Knowledge , 2010, IEEE Transactions on Signal Processing.

[59]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[60]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing, 2nd Edition , 1999 .

[61]  Yaakov Tsaig,et al.  Fast Solution of $\ell _{1}$ -Norm Minimization Problems When the Solution May Be Sparse , 2008, IEEE Transactions on Information Theory.