Automated Tuning of Nonlinear Model Predictive Controller by Reinforcement Learning

One of the major challenges of model predictive control (MPC) for robotic applications is the non-trivial weight tuning process while crafting the objective function. This process is often executed using the trial-and-error method by the user. Consequently, the optimality of the weights and the time required for the process become highly dependent on the skill set and experience of the user. In this study, we present a generic and user-independent framework which automates the tuning process by reinforcement learning. The proposed method shows competency in tuning a nonlinear MPC (NMPC) which is employed for trajectory tracking control of aerial robots. It explores the desirable weights within less than an hour in iterative Gazebo simulations running on a standard desktop computer. The real world experiments illustrate that the NMPC weights explored by the proposed method result in a satisfactory trajectory tracking performance.

[1]  Christos Papachristos,et al.  Dual–Authority Thrust–Vectoring of a Tri–TiltRotor employing Model Predictive Control , 2016, J. Intell. Robotic Syst..

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  Jingxuan Sun,et al.  Model Predictive Control for Path Tracking of a VTOL Tailsitter UAV in an HIL Simulation Environment , 2018 .

[4]  Jan Swevers,et al.  Real-time nonlinear MPC and MHE for a large-scale mechatronic application , 2015 .

[5]  Erdal Kayacan,et al.  Receding horizon control of a 3 DOF helicopter using online estimation of aerodynamic parameters , 2018 .

[6]  Efe Camci,et al.  Waitress quadcopter explores how to serve drinks by reinforcement learning , 2016, 2016 IEEE Region 10 Conference (TENCON).

[7]  Jay H. Lee,et al.  Tuning of model predictive controllers for robust performance , 1994 .

[8]  Erdal Kayacan,et al.  Model Predictive Control in Aerospace Systems: Current State and Opportunities , 2017 .

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Samir Bouabdallah,et al.  Design and control of quadrotors with application to autonomous flying , 2007 .

[11]  Emad Ali,et al.  On-line tuning strategy for model predictive controllers , 2001 .

[12]  E. Ali,et al.  Heuristic On-Line Tuning for Nonlinear Model Predictive Controllers Using Fuzzy Logic , 2003 .

[13]  Suseong Kim,et al.  Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[14]  D. Cooper,et al.  A Tuning Strategy for Unconstrained Multivariable Model Predictive Control , 1998 .

[15]  Girish Chowdhary,et al.  Learning-Based Fast Nonlinear Model Predictive Control for Custom-Made 3D Printed Ground and Aerial Robots , 2018, Handbook of Model Predictive Control.

[16]  Shahram Yousefi,et al.  Experimental results for autonomous model-predictive trajectory planning tuned with machine learning , 2017, 2017 Annual IEEE International Systems Conference (SysCon).

[17]  Marin Kobilarov,et al.  Towards model-predictive control for aerial pick-and-place , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Cairo L. Nascimento,et al.  Design of model predictive control via learning automata for a single UAV load transportation , 2017, 2017 Annual IEEE International Systems Conference (SysCon).