Structured Variational Inference in Partially Observable UnstableGaussian Process State Space Models

We propose a new variational inference algorithm for learning in Gaussian Process State-Space Models (GPSSMs). Our algorithm enables learning of unstable and partially observable systems, where previous algorithms fail. Our main algorithmic contribution is a novel approximate posterior that can be calculated efficiently using a single forward and backward pass along the training trajectories. The forward-backward pass is inspired on Kalman smoothing for linear dynamical systems but generalizes to GPSSMs. Our second contribution is a modification of the conditioning step that effectively lowers the Kalman gain. This modification is crucial to attaining good test performance where no measurements are available. Finally, we show experimentally that our learning algorithm performs well in stable and unstable real systems with hidden states.

[1]  James Hensman,et al.  Identification of Gaussian Process State Space Models , 2017, NIPS.

[2]  David J. Fleet,et al.  Gaussian Process Dynamical Models , 2005, NIPS.

[3]  Alexander G. de G. Matthews,et al.  Scalable Gaussian process inference using variational methods , 2017 .

[4]  Paolo Rapisarda,et al.  Data-driven simulation and control , 2008, Int. J. Control.

[5]  Marc Peter Deisenroth,et al.  Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control , 2017, AISTATS.

[6]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[7]  Neil D. Lawrence,et al.  Gaussian Processes for Big Data , 2013, UAI.

[8]  Sergey Levine,et al.  Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models , 2018, NeurIPS.

[9]  Uri Shalit,et al.  Structured Inference Networks for Nonlinear State Space Models , 2016, AAAI.

[10]  Duy Nguyen-Tuong,et al.  Probabilistic Recurrent State-Space Models , 2018, ICML.

[11]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[12]  Marc Peter Deisenroth,et al.  Doubly Stochastic Variational Inference for Deep Gaussian Processes , 2017, NIPS.

[13]  Il Memming Park,et al.  BLACK BOX VARIATIONAL INFERENCE FOR STATE SPACE MODELS , 2015, 1511.07367.

[14]  Neil D. Lawrence,et al.  Recurrent Gaussian Processes , 2015, ICLR.

[15]  Roland Siegwart,et al.  Towards Efficient Full Pose Omnidirectionality with Overactuated MAVs , 2018, ISER.

[16]  Felix Berkenkamp,et al.  Safe Exploration in Reinforcement Learning: Theory and Applications in Robotics , 2019 .

[17]  A. Doucet,et al.  Monte Carlo Smoothing for Nonlinear Time Series , 2004, Journal of the American Statistical Association.

[18]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[19]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[20]  R. Khasminskii.  Stochastic Stability of Differential Equations , 1980 .

[21]  Carl E. Rasmussen,et al.  Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models , 2019, ICML.

[22]  Stefano Ermon,et al.  Calibrated Model-Based Deep Reinforcement Learning , 2019, ICML.

[23]  Carl E. Rasmussen,et al.  Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC , 2013, NIPS.

[24]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[25]  Kilian Q. Weinberger,et al.  On Calibration of Modern Neural Networks , 2017, ICML.

[26]  N. Bershad,et al.  Random differential equations in science and engineering , 1975, Proceedings of the IEEE.

[27]  Carl E. Rasmussen,et al.  Variational Gaussian Process State-Space Models , 2014, NIPS.