Active Learning in Gaussian Process State Space Model

We investigate active learning in Gaussian Process statespace models (GPSSM). Our problem is to actively steer the system through latent states by determining its inputs such that the underlying dynamics can be optimally learned by a GPSSM. In order that the most informative inputs are selected, we employ mutual information as our active learning criterion. In particular, we present two approaches for the approximation of mutual information for the GPSSM given latent states. The proposed approaches are evaluated in several physical systems where we actively learn the underlying non-linear dynamics represented by the state-space model.

[1]  Andreas Krause,et al.  Safe Model-based Reinforcement Learning with Stability Guarantees , 2017, NIPS.

[2]  Alexis Boukouvalas,et al.  GPflow: A Gaussian Process Library using TensorFlow , 2016, J. Mach. Learn. Res..

[3]  John W. Fisher,et al.  Maximum Mutual Information Principle for Dynamic Sensor Query Problems , 2003, IPSN.

[4]  Sanjay Pant,et al.  A non-parametric k-nearest neighbour entropy estimator , 2016, Physical review. E.

[5]  Alexander A. Alemi,et al.  On Variational Bounds of Mutual Information , 2019, ICML.

[6]  Sebastian Trimpe,et al.  Actively Learning Gaussian Process Dynamics , 2019, L4DC.

[7]  Yarin Gal,et al.  BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning , 2019, NeurIPS.

[8]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[9]  Yong Zeng,et al.  State-Space Models , 2013 .

[10]  Carl E. Rasmussen,et al.  Closed-form Inference and Prediction in Gaussian Process State-Space Models , 2018, ArXiv.

[11]  Klaus Obermayer,et al.  Gaussian Process Regression: Active Data Selection and Test Point Rejection , 2000, DAGM-Symposium.

[12]  Zoubin Ghahramani,et al.  Bayesian Active Learning for Classification and Preference Learning , 2011, ArXiv.

[13]  Donald E. Kirk,et al.  Optimal control theory : an introduction , 1970 .

[14]  Niall Twomey,et al.  Bayesian Active Learning with Evidence-Based Instance Selection , 2015 .

[15]  Farhan A. Faruqi.  State Space Model for Autopilot Design of Aerospace Vehicles , 2007 .

[16]  Jonathan P. How,et al.  Sample Efficient Reinforcement Learning with Gaussian Processes , 2014, ICML.

[17]  Martin Kliesch,et al.  On products of Gaussian random variables , 2017, 1711.10516.

[18]  Shie Mannor,et al.  Reinforcement learning with Gaussian processes , 2005, ICML.

[19]  Mihaela van der Schaar,et al.  Attentive State-Space Modeling of Disease Progression , 2019, NeurIPS.

[20]  Petko H. Petkov,et al.  Robust Real-Time Control of a Two-Rotor Aerodynamic System , 2008 .

[21]  Changjian Shui,et al.  Deep Active Learning: Unified and Principled Method for Query and Training , 2020, AISTATS.

[22]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[23]  Dieter Fox,et al.  GP-BayesFilters: Bayesian filtering using Gaussian process prediction and observation models , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  M. Springer,et al.  The Distribution of Products of Beta, Gamma and Gaussian Random Variables , 1970 .

[25]  Fernando Pérez-Cruz,et al.  Estimation of Information Theoretic Measures for Continuous Random Variables , 2008, NIPS.

[26]  Matthias W. Seeger,et al.  Deep State Space Models for Time Series Forecasting , 2018, NeurIPS.

[27]  Wenbin Cai,et al.  Batch Mode Active Learning for Regression With Expected Model Change , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[29]  Alexander A. Alemi,et al.  Fixing a Broken ELBO , 2017, ICML.

[30]  Andreas Krause,et al.  Nonmyopic active learning of Gaussian processes: an exploration-exploitation approach , 2007, ICML '07.

[31]  Razvan V. Florian,et al.  Correct equations for the dynamics of the cart-pole system , 2005 .

[32]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[33]  James Hensman,et al.  Identification of Gaussian Process State Space Models , 2017, NIPS.

[34]  Zoubin Ghahramani,et al.  Deep Bayesian Active Learning with Image Data , 2017, ICML.

[35]  James Hensman,et al.  On Sparse Variational Methods and the Kullback-Leibler Divergence between Stochastic Processes , 2015, AISTATS.

[36]  Duy Nguyen-Tuong,et al.  Safe Active Learning for Time-Series Modeling with Gaussian Processes , 2018, NeurIPS.

[37]  Carl E. Rasmussen,et al.  Variational Gaussian Process State-Space Models , 2014, NIPS.

[38]  Neil D. Lawrence,et al.  Latent Autoregressive Gaussian Processes Models for Robust System Identification , 2016 .

[39]  Sandra Hirche,et al.  Localized active learning of Gaussian process state space models , 2020, L4DC.

[40]  Carl E. Rasmussen,et al.  Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC , 2013, NIPS.

[41]  Karl Stratos,et al.  Formal Limitations on the Measurement of Mutual Information , 2018, AISTATS.

[42]  Corinna Cortes,et al.  Understanding the Effects of Batching in Online Active Learning , 2020, AISTATS.

[43]  David J. Fleet,et al.  Gaussian Process Dynamical Models , 2005, NIPS.

[44]  S. Kakade,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2012, IEEE Transactions on Information Theory.

[45]  Bernard Friedland,et al.  Control System Design: An Introduction to State-Space Methods , 1987 .

[46]  Carl E. Rasmussen,et al.  State-Space Inference and Learning with Gaussian Processes , 2010, AISTATS.

[47]  Duy Nguyen-Tuong,et al.  Safe Exploration for Active Learning with Gaussian Processes , 2015, ECML/PKDD.

[48]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[49]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[50]  Pieter Abbeel,et al.  Mutual Information Maximization for Robust Plannable Representations , 2019, ArXiv.

[51]  James D. Hamilton.  State-space models , 1994 .

[52]  Sanjoy Dasgupta,et al.  Two faces of active learning , 2011, Theor. Comput. Sci..

[53]  Roger Frigola,et al.  Bayesian Time Series Learning with Gaussian Processes , 2015 .

[54]  Duy Nguyen-Tuong,et al.  Probabilistic Recurrent State-Space Models , 2018, ICML.

[55]  Sebastian Trimpe,et al.  Joint State and Dynamics Estimation With High-Gain Observers and Gaussian Process Models , 2021, IEEE Control Systems Letters.