Scenario-based Optimal Control for Gaussian Process State Space Models

Data-driven approaches from machine learning provide powerful tools to identify dynamical systems with limited prior knowledge of the model structure. More particular, the Gaussian process state space model, a Bayesian nonparametric approach, is increasingly utilized in control. Its probabilistic nature is interpreted differently in the control literature, but so far, it is not considered as a distribution over dynamical system which allows a scenario-based control design. This paper introduces how scenarios are sampled from a Gaussian process and utilizes them in a differential dynamic programming approach to solve an optimal control problem. For the linear-quadratic case, we derive probabilistic performance guarantees using results from robust convex optimization. The proposed methods are evaluated numerically for the nonlinear and linear case.

[1]  M. Campi,et al.  The scenario approach for systems and control design , 2008 .

[2]  Yunpeng Pan,et al.  Belief space stochastic control under unknown dynamics , 2017, 2017 American Control Conference (ACC).

[3]  Sandra Hirche,et al.  Gaussian processes for dynamic movement primitives with application in knowledge-based cooperation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Giuseppe Carlo Calafiore,et al.  The scenario approach to robust control design , 2006, IEEE Transactions on Automatic Control.

[5]  Juš Kocijan,et al.  Modelling and Control of Dynamic Systems Using Gaussian Process Models , 2015 .

[6]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[7]  Marco C. Campi,et al.  Why Is Resorting to Fate Wise? A Critical Look at Randomized Algorithms in Systems and Control , 2010, Eur. J. Control.

[8]  Sandra Hirche,et al.  Learning stable Gaussian process state space models , 2017, 2017 American Control Conference (ACC).

[9]  C. Karen Liu,et al.  Differential dynamic programming with nonlinear constraints , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Giuseppe Carlo Calafiore,et al.  Robust Model Predictive Control via Scenario Optimization , 2012, IEEE Transactions on Automatic Control.

[11]  D. Mayne.  A Second-order Gradient Method for Determining Optimal Trajectories of Non-linear Discrete-time Systems , 1966 .

[12]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[13]  Sandra Hirche,et al.  Bayesian uncertainty modeling for programming by demonstration , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[14]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[15]  D. Wolpert.  The Supervised Learning No-Free-Lunch Theorems , 2002 .

[16]  Sandra Hirche,et al.  Stable Model-based Control with Gaussian Process Regression for Robot Manipulators , 2018, ArXiv.

[17]  Giorgio Gnecco,et al.  Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty , 2014, Comput. Optim. Appl..

[18]  Sandra Hirche,et al.  Feedback linearization using Gaussian processes , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[19]  Dana Kulic,et al.  Stable Gaussian process based tracking control of Lagrangian systems , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).