A Hierarchical Bayesian Linear Regression Model with Local Features for Stochastic Dynamics Approximation

One of the challenges in model-based control of stochastic dynamical systems is that the state transition dynamics are involved, and it is not easy or efficient to make good-quality predictions of the states. Moreover, there are not many representational models for the majority of autonomous systems, as it is not easy to build a compact model that captures the entire dynamical subtleties and uncertainties. In this work, we present a hierarchical Bayesian linear regression model with local features to learn the dynamics of a micro-robotic system as well as two simpler examples, consisting of a stochastic mass-spring damper and a stochastic double inverted pendulum on a cart. The model is hierarchical since we assume non-stationary priors for the model parameters. These non-stationary priors make the model more flexible by imposing priors on the priors of the model. To solve the maximum likelihood (ML) problem for this hierarchical model, we use the variational expectation maximization (EM) algorithm, and enhance the procedure by introducing hidden target variables. The algorithm yields parsimonious model structures, and consistently provides fast and accurate predictions for all our examples involving large training and test sets. This demonstrates the effectiveness of the method in learning stochastic dynamics, which makes it suitable for future use in a paradigm, such as model-based reinforcement learning, to compute optimal control policies in real time.

[1]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[2]  Geoffrey E. Hinton,et al.  An Alternative Model for Mixtures of Experts , 1994, NIPS.

[3]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[4]  Carl E. Rasmussen,et al.  Convolutional Gaussian Processes , 2017, NIPS.

[5]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[6]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[7]  Giovanni Volpe,et al.  Computational toolbox for optical tweezers in geometrical optics , 2014, 1402.5439.

[8]  Steven L. Brunton,et al.  On dynamic mode decomposition: Theory and applications , 2013, 1312.0041.

[9]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[10]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[11]  Mehran Mesbahi,et al.  Linear Model Regression on Time-series Data: Non-asymptotic Error Bounds and Applications , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[12]  Carl E. Rasmussen,et al.  Sparse Spectrum Gaussian Process Regression , 2010, J. Mach. Learn. Res..

[13]  Peter Grünwald,et al.  A Tight Excess Risk Bound via a Unified PAC-Bayesian-Rademacher-Shtarkov-MDL Complexity , 2017, ALT.

[14]  Christopher G. Atkeson,et al.  Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming , 1993, NIPS.

[15]  James Hensman,et al.  On Sparse Variational Methods and the Kullback-Leibler Divergence between Stochastic Processes , 2015, AISTATS.

[16]  Francesco Corona,et al.  Regional models: A new approach for nonlinear system identification via clustering of the self-organizing map , 2015, Neurocomputing.

[17]  Ashis Gopal Banerjee,et al.  A Step Toward Learning to Control Tens of Optically Actuated Microrobots in Three Dimensions , 2018, 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE).

[18]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[19]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[20]  T. Hastie,et al.  Local Regression: Automatic Kernel Carpentry , 1993 .

[21]  James Hensman,et al.  MCMC for Variationally Sparse Gaussian Processes , 2015, NIPS.

[22]  Igor Kononenko,et al.  Bayesian neural networks , 1989, Biological Cybernetics.

[23]  Han-Lim Choi,et al.  Multiscale abstraction, planning and control using diffusion wavelets for stochastic optimal control problems , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Ashis Gopal Banerjee,et al.  Toward automated formation of microsphere arrangements using multiplexed optical tweezers , 2016, NanoScience + Engineering.

[25]  Maya R. Gupta,et al.  Theory and Use of the EM Algorithm , 2011, Found. Trends Signal Process..

[26]  Alexandre Lacoste,et al.  PAC-Bayesian Theory Meets Bayesian Inference , 2016, NIPS.

[27]  Lehel Csató,et al.  Sparse On-Line Gaussian Processes , 2002, Neural Computation.

[28]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[29]  Alexander G. de G. Matthews,et al.  Scalable Gaussian process inference using variational methods , 2017 .

[30]  Ron Meir,et al.  Generalization Error Bounds for Bayesian Mixture Algorithms , 2003, J. Mach. Learn. Res..

[31]  Stefan Schaal,et al.  Locally Weighted Regression for Control , 2010 .

[32]  Scott Fortmann-Roe,et al.  Understanding the bias-variance tradeoff , 2012 .

[33]  R. A. Boyles On the Convergence of the EM Algorithm , 1983 .

[34]  Stefan Schaal,et al.  Locally Weighted Projection Regression : An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space , 2000 .

[35]  Manfred Opper,et al.  The Variational Gaussian Approximation Revisited , 2009, Neural Computation.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Steven L. Brunton,et al.  Data-Driven Identification of Parametric Partial Differential Equations , 2018, SIAM J. Appl. Dyn. Syst..

[38]  Michael I. Jordan Learning in Graphical Models , 1999, NATO ASI Series.

[39]  Christopher G. Atkeson,et al.  Using Local Models to Control Movement , 1989, NIPS.

[40]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[41]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[42]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[43]  Andrew Gordon Wilson,et al.  Stochastic Variational Deep Kernel Learning , 2016, NIPS.

[44]  Christopher G. Atkeson,et al.  Nonparametric Model-Based Reinforcement Learning , 1997, NIPS.

[45]  Giorgio Metta,et al.  Real-time model learning using Incremental Sparse Spectrum Gaussian Process Regression. , 2013, Neural networks : the official journal of the International Neural Network Society.

[46]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[47]  O. Nelles Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models , 2000 .

[48]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[49]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[50]  D.G. Tzikas,et al.  The variational approximation for Bayesian inference , 2008, IEEE Signal Processing Magazine.

[51]  Alexander Y. Bogdanov,et al.  Optimal Control of a Double Inverted Pendulum on a Cart , 2004 .

[52]  William J. Byrne,et al.  Convergence Theorems for Generalized Alternating Minimization Procedures , 2005, J. Mach. Learn. Res..

[53]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[54]  Vladimir Cherkassky,et al.  Model complexity control for regression using VC generalization bounds , 1999, IEEE Trans. Neural Networks.

[55]  Babatunde A. Ogunnaike,et al.  Process Dynamics, Modeling, and Control , 1994 .

[56]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[57]  Martin J. Wainwright,et al.  Statistical guarantees for the EM algorithm: From population to sample-based analysis , 2014, ArXiv.

[58]  A. F. Adams,et al.  The Survey , 2021, Dyslexia in Higher Education.

[59]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[60]  Stefan Schaal,et al.  Efficient Bayesian local model learning for control , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[61]  Ambuj Tewari,et al.  On the Complexity of Linear Prediction: Risk Bounds, Margin Bounds, and Regularization , 2008, NIPS.

[62]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[63]  Manfred Opper,et al.  General Bounds on Bayes Errors for Regression with Gaussian Processes , 1998, NIPS.