Sparse Bayesian Learning With Dynamic Filtering for Inference of Time-Varying Sparse Signals

Many signal processing applications require estimation of time-varying sparse signals, potentially with the knowledge of an imperfect dynamics model. In this paper, we propose an algorithm for dynamic filtering of time-varying sparse signals based on the sparse Bayesian learning (SBL) framework. The key idea underlying the algorithm, termed SBL-DF, is the incorporation of a signal prediction generated from a dynamics model and estimates of previous time steps into the hyperpriors of the SBL probability model. The proposed algorithm is online, robust to imperfect dynamics models (due to the propagation of dynamics information through higher-order statistics), robust to certain undesirable dictionary properties such as coherence (due to properties of the SBL framework), allows the use of arbitrary dynamics models, and requires the tuning of fewer parameters than many other dynamic filtering algorithms do. We also extend the fast marginal likelihood SBL inference procedure to the informative hyperprior setting to create a particularly efficient version of the SBL-DF algorithm. Numerical simulations show that SBL-DF converges much faster and to more accurate solutions than standard SBL and other dynamical filtering algorithms. In particular, we show that SBL-DF outperforms state of the art algorithms when the dictionary contains the challenging coherence and column scaling structure found in many practical applications.

[1]  Julia P. Owen,et al.  Robust Bayesian estimation of the location, orientation, and time course of multiple correlated neural sources using MEG , 2010, NeuroImage.

[2]  C. Striebel,et al.  On the maximum likelihood estimates for linear dynamic systems , 1965 .

[3]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[4]  Dave Zachariah,et al.  Dynamic Iterative Pursuit , 2012, IEEE Transactions on Signal Processing.

[5]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[6]  Srikantan S. Nagarajan,et al.  Iterative Reweighted l1 and l2 Methods for Finding Sparse Solution , 2016 .

[7]  H. Vincent Poor,et al.  Fast adaptive variational sparse Bayesian learning with automatic relevance determination , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Bhaskar D. Rao,et al.  Latent Variable Bayesian Models for Promoting Sparsity , 2011, IEEE Transactions on Information Theory.

[9]  Justin K. Romberg,et al.  Sparsity penalties in dynamical system estimation , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[10]  Christopher J. Rozell,et al.  Earth Mover's Distance as a Dynamics Regularizer for Sparse Signal Tracking , 2018 .

[11]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[12]  Todd K. Moon,et al.  Sparse Bayesian learning boosted by partial erroneous support knowledge , 2016, 2016 50th Asilomar Conference on Signals, Systems and Computers.

[13]  Stergios I. Roumeliotis,et al.  Lasso-Kalman smoother for tracking sparse signals , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[14]  David P. Wipf,et al.  Sparse Estimation with Structured Dictionaries , 2011, NIPS.

[15]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[16]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[17]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[18]  Wen Gao,et al.  Building Invariances Into Sparse Subspace Clustering , 2018, IEEE Transactions on Signal Processing.

[19]  Bhaskar D. Rao,et al.  An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem , 2007, IEEE Transactions on Signal Processing.

[20]  Zhilin Zhang,et al.  Exploiting Correlation in Sparse Signal Recovery Problems: Multiple Measurement Vectors, Block Sparsity, and Time-Varying Sparsity , 2011, ArXiv.

[21]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Bhaskar D. Rao,et al.  Perspectives on Sparse Bayesian Learning , 2003, NIPS.

[23]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[24]  David P. Wipf,et al.  Exploiting the convex-concave penalty for tracking: A novel dynamic reweighted sparse Bayesian learning algorithm , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[26]  Peter Gerstoft,et al.  Sparse Bayesian learning with multiple dictionaries , 2017, Signal Process..

[27]  Chandra R. Murthy,et al.  A Noniterative Online Bayesian Algorithm for the Recovery of Temporally Correlated Sparse Vectors , 2017, IEEE Transactions on Signal Processing.

[28]  Bhaskar D. Rao,et al.  A GAMP-Based Low Complexity Sparse Bayesian Learning Algorithm , 2017, IEEE Transactions on Signal Processing.

[29]  David P. Wipf,et al.  A New View of Automatic Relevance Determination , 2007, NIPS.

[30]  Namrata Vaswani,et al.  LS-CS-Residual (LS-CS): Compressive Sensing on Least Squares Residual , 2009, IEEE Transactions on Signal Processing.

[31]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[32]  Wei Dai,et al.  A fast variational approach for Bayesian compressive sensing with informative priors , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Wei Dai,et al.  Tracking dynamic sparse signals with hierarchical Kalman filters: A case study , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[34]  Zhi Chen,et al.  Support knowledge-aided sparse Bayesian learning for compressed sensing , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  Christopher J. Rozell,et al.  Dynamic Filtering of Time-Varying Sparse Signals via $\ell _1$ Minimization , 2015, IEEE Transactions on Signal Processing.

[36]  Thomas Buchgraber,et al.  Variational Sparse Bayesian Learning: Centralized and Distributed Processing , 2013 .

[37]  Bhaskar D. Rao,et al.  Sparse Signal Recovery With Temporally Correlated Source Vectors Using Sparse Bayesian Learning , 2011, IEEE Journal of Selected Topics in Signal Processing.

[38]  David P. Wipf,et al.  From Bayesian Sparsity to Gated Recurrent Nets , 2017, NIPS.

[39]  Georgios B. Giannakis,et al.  Doubly Robust Smoothing of Dynamical Processes via Outlier Sparsity Constraints , 2011, IEEE Transactions on Signal Processing.

[40]  Bruno A. Olshausen,et al.  Group Sparse Coding with a Laplacian Scale Mixture Prior , 2010, NIPS.

[41]  H. Vincent Poor,et al.  A sliding-window online fast variational sparse Bayesian learning algorithm , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Peter Gerstoft,et al.  Sparse Bayesian learning with multiple dictionaries , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[43]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[44]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[45]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[46]  Kin K. Leung,et al.  Tracking dynamic sparse signals using Hierarchical Bayesian Kalman filters , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[47]  Zhilin Zhang Sparse signal recovery exploiting spatiotemporal correlation , 2012 .

[48]  Peter Boesiger,et al.  Compressed sensing in dynamic MRI , 2008, Magnetic resonance in medicine.

[49]  Pini Gurfil,et al.  Methods for Sparse Signal Recovery Using Kalman Filtering With Embedded Pseudo-Measurement Norms and Quasi-Norms , 2010, IEEE Transactions on Signal Processing.

[50]  Christoph F. Mecklenbräuker,et al.  Sequential Bayesian Sparse Signal Reconstruction Using Array Data , 2013, IEEE Transactions on Signal Processing.

[51]  E.J. Candes Compressive Sampling , 2022 .

[52]  Marian Codreanu,et al.  A Bayesian Approach for Online Recovery of Streaming Signals From Compressive Measurements , 2017, IEEE Transactions on Signal Processing.

[53]  Justin K. Romberg,et al.  Sparse Recovery of Streaming Signals Using $\ell_1$-Homotopy , 2013, IEEE Transactions on Signal Processing.

[54]  Namrata Vaswani,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, ISIT.

[55]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[56]  Justin K. Romberg,et al.  Estimation and dynamic updating of time-varying signals with sparse variations , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[57]  Aleksandr Y. Aravkin,et al.  Sparse/robust estimation and Kalman smoothing with nonsmooth log-concave densities: modeling, computation, and theory , 2013, J. Mach. Learn. Res..

[58]  Christopher J. Rozell,et al.  Sparse Dynamic Filtering via Earth Mover's Distance Regularization , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).