Online Variational Filtering and Parameter Learning

We present a variational method for online state estimation and parameter learning in state-space models (SSMs), a ubiquitous class of latent variable models for sequential data. As per standard batch variational techniques, we use stochastic gradients to simultaneously optimize a lower bound on the log evidence with respect to both model parameters and a variational approximation of the states’ posterior distribution. However, unlike existing approaches, our method is able to operate in an entirely online manner, such that historic observations do not require revisitation after being incorporated and the cost of updates at each time step remains constant, despite the growing dimensionality of the joint posterior distribution of the states. This is achieved by utilizing backward decompositions of this joint posterior distribution and of its variational approximation, combined with Bellman-type recursions for the evidence lower bound and its gradients. We demonstrate the performance of this methodology across several examples, including high-dimensional SSMs and sequential Variational Auto-Encoders.

[1]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[2]  Timo Gerkmann,et al.  Speech Enhancement with Stochastic Temporal Convolutional Networks , 2020, INTERSPEECH.

[3]  Arnaud Doucet,et al.  Asymptotic Properties of Recursive Particle Maximum Likelihood Estimation , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[4]  Arnaud Doucet,et al.  Differentiable Particle Filtering via Entropy-Regularized Optimal Transport , 2021, ICML.

[5]  Thomas B. Schön,et al.  Variational State and Parameter Estimation , 2020, IFAC-PapersOnLine.

[6]  V B Tadić,et al.  Analyticity, Convergence, and Convergence Rate of Recursive Maximum-Likelihood Estimation in Hidden Markov Models , 2009, IEEE Transactions on Information Theory.

[7]  Simo Särkkä,et al.  Bayesian Filtering and Smoothing , 2013, Institute of Mathematical Statistics textbooks.

[8]  Kyunghyun Cho,et al.  A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks , 2019, J. Mach. Learn. Res..

[9]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[10]  Karol Gregor,et al.  Temporal Difference Variational Auto-Encoder , 2018, ICLR.

[11]  Tuan Anh Le,et al.  Auto-Encoding Sequential Monte Carlo , 2017, ICLR.

[12]  Yuan Zhao,et al.  Streaming Variational Monte Carlo , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  P. Bickel,et al.  Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems , 2008, 0805.3034.

[14]  Matthew Fellows,et al.  VIREL: A Variational Inference Framework for Reinforcement Learning , 2018, NeurIPS.

[15]  Ole Winther,et al.  Sequential Neural Models with Stochastic Layers , 2016, NIPS.

[16]  Bernard Hanzon,et al.  A differential geometric approach to nonlinear filtering: the projection filter , 1998, IEEE Trans. Autom. Control..

[17]  G. Evensen.  Data Assimilation: The Ensemble Kalman Filter , 2006 .

[18]  Yee Whye Teh,et al.  Filtering Variational Objectives , 2017, NIPS.

[19]  Uri Shalit,et al.  Structured Inference Networks for Nonlinear State Space Models , 2016, AAAI.

[20]  Yisong Yue,et al.  A General Method for Amortizing Variational Filtering , 2018, NeurIPS.

[21]  Oliver Brock,et al.  Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors , 2018, Robotics: Science and Systems.

[22]  David Silver,et al.  Reinforced Variational Inference , 2015, NIPS 2015.

[23]  Sergey Levine,et al.  Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review , 2018, ArXiv.

[24]  Il Memming Park,et al.  BLACK BOX VARIATIONAL INFERENCE FOR STATE SPACE MODELS , 2015, 1511.07367.

[25]  Hongseok Yang,et al.  Variational Inference for Sequential Data with Future Likelihood Estimates , 2020, ICML.

[26]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[27]  Ba-Ngu Vo,et al.  A Solution for Large-Scale Multi-Object Tracking , 2018, IEEE Transactions on Signal Processing.

[28]  Il Memming Park,et al.  Variational Online Learning of Neural Dynamics , 2017, Frontiers in Computational Neuroscience.

[29]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[30]  L. Gerencsér,et al.  Recursive estimation of Hidden Markov Models , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[31]  Tom Ryder,et al.  The neural moving average model for scalable variational inference of state space models , 2019, UAI.

[32]  P. S. Krishnaprasad,et al.  Approximate nonlinear filtering and its application in navigation , 2005, Autom..

[33]  Stephan Mandt,et al.  Disentangled Sequential Autoencoder , 2018, ICML.

[34]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[35]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[36]  Arnaud Doucet,et al.  On Particle Methods for Parameter Estimation in State-Space Models , 2014, 1412.8695.

[37]  Kazufumi Ito,et al.  Gaussian filters for nonlinear filtering problems , 2000, IEEE Trans. Autom. Control..

[38]  Maneesh Sahani,et al.  Amortised Learning by Wake-Sleep , 2020, ICML.

[39]  Randal Douc,et al.  Nonlinear Time Series: Theory, Methods and Applications with R Examples , 2014 .

[40]  Václav Smídl,et al.  Variational Bayesian Filtering , 2008, IEEE Transactions on Signal Processing.

[41]  Scott W. Linderman,et al.  Variational Sequential Monte Carlo , 2017, AISTATS.

[42]  David Hsu,et al.  Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations , 2020, ICLR.

[43]  Eric R. Ziegel,et al.  Analysis of Financial Time Series , 2002, Technometrics.