Cautious Model Predictive Control Using Gaussian Process Regression

Gaussian process (GP) regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in function estimation. In the context of control, it is seeing increasing use for modeling of nonlinear dynamical systems from data, as it allows the direct assessment of residual model uncertainty. We present a model predictive control (MPC) approach that integrates a nominal system with an additive nonlinear part of the dynamics modeled as a GP. We describe a principled way of formulating the chance-constrained MPC problem, which takes into account residual uncertainties provided by the GP model to enable cautious control. Using additional approximations for efficient computation, we finally demonstrate the approach in a simulation example, as well as in a hardware implementation for autonomous racing of remote-controlled race cars with fast sampling times of 20 ms, highlighting improvements with regard to both performance and safety over a nominal controller.

[1]  Lukas Hewing,et al.  Stochastic Model Predictive Control for Linear Systems Using Probabilistic Reachable Sets , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[2]  Marc Peter Deisenroth,et al.  Efficient reinforcement learning using Gaussian processes , 2010 .

[3]  Hans B. Pacejka,et al.  THE MAGIC FORMULA TYRE MODEL , 1991 .

[4]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[5]  Manfred Morari,et al.  Optimization‐based autonomous racing of 1:43 scale RC cars , 2015, ArXiv.

[6]  Lennart Ljung,et al.  Kernel methods in system identification, machine learning and function estimation: A survey , 2014, Autom..

[7]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[8]  Hans Joachim Ferreau,et al.  Efficient Numerical Methods for Nonlinear MPC and Moving Horizon Estimation , 2009 .

[9]  Chris Manzie,et al.  Model predictive contouring control , 2010, 49th IEEE Conference on Decision and Control (CDC).

[10]  A. Mesbah,et al.  Stochastic Model Predictive Control: An Overview and Perspectives for Future Research , 2016, IEEE Control Systems.

[11]  Volker Tresp,et al.  A Bayesian Committee Machine , 2000, Neural Computation.

[12]  Malte Kuß,et al.  Gaussian process models for robust regression, classification, and reinforcement learning , 2006 .

[13]  Lukas Hewing,et al.  On a Correspondence between Probabilistic and Robust Invariant Sets for Linear Systems , 2018 .

[14]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[15]  P. Whittle.  Risk-sensitive linear/quadratic/gaussian control , 1981, Advances in Applied Probability.

[16]  Joel Andersson,et al.  A General-Purpose Software Framework for Dynamic Optimization (Een algemene softwareomgeving voor dynamische optimalisatie) , 2013 .

[17]  Marc Peter Deisenroth,et al.  Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control , 2017, AISTATS.

[18]  Alberto Bemporad,et al.  Robust model predictive control: A survey , 1998, Robustness in Identification and Control.

[19]  Angela P. Schoellig,et al.  Learning-based nonlinear model predictive control to improve vision-based mobile robot path-tracking in challenging outdoor environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Manfred Morari,et al.  A tractable approximation of chance constrained stochastic MPC based on affine disturbance feedback , 2008, 2008 47th IEEE Conference on Decision and Control.

[21]  Roderick Murray-Smith,et al.  Gaussian Process priors with Uncertain Inputs: Multiple-Step-Ahead Prediction , 2002 .

[22]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[23]  Fakhrul Alam,et al.  Gaussian Process Model Predictive Control of Unknown Nonlinear Systems , 2016, ArXiv.

[24]  Uwe D. Hanebeck,et al.  Analytic moment-based Gaussian process filtering , 2009, ICML '09.

[25]  J. Kocijan,et al.  Gaussian process model based predictive control , 2004, Proceedings of the 2004 American Control Conference.

[26]  Basil Kouvaritakis,et al.  Stochastic tubes in model predictive control with probabilistic constraints , 2010, Proceedings of the 2010 American Control Conference.

[27]  Christopher D. McKinnon,et al.  Experience-Based Model Selection to Enable Long-Term, Safe Control for Repetitive Tasks Under Changing Conditions , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Carl E. Rasmussen,et al.  Sparse Spectrum Gaussian Process Regression , 2010, J. Mach. Learn. Res..

[29]  Eduardo F. Camacho,et al.  Min-max Model Predictive Control of Nonlinear Systems: A Unifying Overview on Stability , 2009, Eur. J. Control.

[30]  Zhou Zhou,et al.  Reliable approximations of probability-constrained stochastic linear-quadratic control , 2013, Autom..

[31]  Alexander Domahidi,et al.  FORCES NLP: an efficient implementation of interior-point methods for multistage nonlinear nonconvex programs , 2020, Int. J. Control.

[32]  O. Bosgra,et al.  Closed-loop stochastic dynamic process optimization under input and state constraints , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[33]  Alexander Liniger,et al.  Cautious NMPC with Gaussian Process Dynamics for Miniature Race Cars , 2017, ArXiv.

[34]  Angela P. Schoellig,et al.  Robust Constrained Learning-based NMPC enabling reliable mobile robot path tracking , 2016, Int. J. Robotics Res..

[35]  Lukas Hewing,et al.  On a Correspondence between Probabilistic and Robust Invariant Sets for Linear Systems , 2018, 2018 European Control Conference (ECC).

[36]  Jorge Nocedal,et al.  Knitro: An Integrated Package for Nonlinear Optimization , 2006 .

[37]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[38]  Frank Allgöwer,et al.  Robust model predictive control for nonlinear discrete‐time systems , 2003 .

[39]  Olivier Sigaud,et al.  On-line regression algorithms for learning mechanical models of robots: A survey , 2011, Robotics Auton. Syst..

[40]  Ye Wang,et al.  Stochastic model predictive control based on Gaussian processes applied to drinking water networks , 2016 .

[41]  Agathe Girard,et al.  Prediction at an Uncertain Input for Gaussian Processes and Relevance Vector Machines Application to Multiple-Step Ahead Time-Series Forecasting , 2002 .

[42]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[43]  Sahjendra N. Singh,et al.  State-dependent Riccati equation-based robust dive plane control of AUV with control constraints , 2007 .

[44]  Torsten Koller,et al.  Learning-Based Model Predictive Control for Safe Exploration , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[45]  J. Weston,et al.  Approximation Methods for Gaussian Process Regression , 2007 .

[46]  Moritz Diehl,et al.  ACADO toolkit—An open‐source framework for automatic control and dynamic optimization , 2011 .

[47]  John Lygeros,et al.  Efficient implementation of Randomized MPC for miniature race cars , 2016, 2016 European Control Conference (ECC).

[48]  Manfred Morari,et al.  Linear controller design for chance constrained systems , 2014, Autom..

[49]  J. Cunningham,et al.  Gaussian Probabilities and Expectation Propagation , 2011, 1111.6832.

[50]  José A. De Doná,et al.  Probabilistic set invariance and ultimate boundedness , 2012, Autom..

[51]  Fakhrul Alam,et al.  Gaussian Process Model Predictive Control of unmanned quadrotors , 2016, 2016 2nd International Conference on Control, Automation and Robotics (ICCAR).

[52]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[53]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[54]  Xiaoke Yang,et al.  Fault tolerant control using Gaussian processes and model predictive control , 2013 .

[55]  Angela P. Schoellig,et al.  Learning‐based Nonlinear Model Predictive Control to Improve Vision‐based Mobile Robot Path Tracking , 2016, J. Field Robotics.

[56]  Byron Boots,et al.  Prediction under Uncertainty in Sparse Spectrum Gaussian Processes with Applications to Filtering and Control , 2017, ICML.

[57]  Tor Arne Johansen,et al.  Explicit stochastic predictive control of combustion plants based on Gaussian process models , 2008, Autom..

[58]  Bernhard Schölkopf,et al.  Gaussian Process-Based Predictive Control for Periodic Error Correction , 2016, IEEE Transactions on Control Systems Technology.