Learning-based Approximate Model Predictive Control with Guarantees: Joining Neural Networks with Recent Robust MPC

In this work, we present a model predictive control (MPC) method for applications in complex constrained physical systems. We base our work on a novel robust model predictive control (RMPC) scheme guaranteeing constraint satisfaction and recursive feasibility under disturbances. The used scheme keeps the computational complexity comparable to the nominal case. We adopt this approach and extend it by practical useful additions, such as robust dynamic set point tracking and the handling of nonlinear constraints in the output function. We are able to demonstrate this method’s practical significance by controlling a complex robotic system (MPI Apollo robot). Furthermore we tackle the inherent computational complexity of the optimization problem by approximating the RMPC controller with neural network (NN) regression. By only using the NN, we are able to control the robot in a much more dynamic way due to a computational speed-up of several magnitudes. Finally, we also analyze the quality of the approximation by providing closed loop statistical guarantees for the NN controller, also taking into account additional experimentally validated non-idealities of our model. Most of the code, including the TensorFlow NN inference, is fully written in C++ and available on the Max Planck Autonomous Motion Department Gitlab server.

[1]  Yoshua Bengio,et al.  Tackling Climate Change with Machine Learning , 2019, ACM Comput. Surv..

[2]  J. Romm Climate Change: What Everyone Needs to Know® , 2015 .

[3]  David Q. Mayne,et al.  Robustifying model predictive control of constrained linear systems , 2001 .

[4]  D. Limón,et al.  Robust MPC of constrained nonlinear systems based on interval arithmetic , 2005 .

[5]  Eduardo F. Camacho,et al.  Robust tube-based MPC for tracking of constrained linear systems with additive disturbances , 2010 .

[6]  Frank Allgöwer,et al.  A Computationally Efficient Robust Model Predictive Control Framework for Uncertain Nonlinear Systems , 2019, IEEE Transactions on Automatic Control.

[7]  Manfred Morari,et al.  Auto-generated algorithms for nonlinear model predictive control on long and on short horizons , 2013, 52nd IEEE Conference on Decision and Control.

[8]  Frank Allgöwer,et al.  Collision avoidance for uncertain nonlinear systems with moving obstacles using robust Model Predictive Control , 2019, 2019 18th European Control Conference (ECC).

[9]  Sergey Levine,et al.  Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics , 2014, NIPS.

[10]  David Q. Mayne,et al.  Robust model predictive control of constrained linear systems with bounded disturbances , 2005, Autom..

[11]  Bruno Siciliano,et al.  Kinematic control of redundant robot manipulators: A tutorial , 1990, J. Intell. Robotic Syst..

[12]  Tamim Asfour,et al.  An integrated approach to inverse kinematics and path planning for redundant manipulators , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[13]  Dmitry Berenson,et al.  Humanoid motion planning for dual-arm manipulation and re-grasping tasks , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Benjamin Karg,et al.  A deep learning-based approach to robust nonlinear model predictive control , 2018 .

[15]  Sergey Levine,et al.  PLATO: Policy learning using adaptive trajectory optimization , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Moritz Diehl,et al.  An auto-generated real-time iteration algorithm for nonlinear MPC in the microsecond range , 2011, Autom..

[17]  Eduardo F. Camacho,et al.  Min-max Model Predictive Control of Nonlinear Systems: A Unifying Overview on Stability , 2009, Eur. J. Control.

[18]  Sergey Levine,et al.  Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Frank Allgöwer,et al.  Nonlinear Reference Tracking: An Economic Model Predictive Control Perspective , 2019, IEEE Transactions on Automatic Control.

[20]  Benjamin Karg,et al.  Efficient Representation and Approximation of Model Predictive Control Laws via Deep Learning , 2018, IEEE Transactions on Cybernetics.

[21]  E. Yaz Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[22]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[23]  Frank Allgöwer,et al.  A Nonlinear Model Predictive Control Framework Using Reference Generic Terminal Ingredients , 2019, IEEE Transactions on Automatic Control.

[24]  Roy S. Smith,et al.  Parameter Identification of the KUKA LBR iiwa Robot Including Constraints on Physical Feasibility , 2017 .

[25]  Mi-Ching Tsai,et al.  Robust and Optimal Control , 2014 .

[26]  Moritz Diehl,et al.  ACADO toolkit—An open‐source framework for automatic control and dynamic optimization , 2011 .

[27]  Jonas Buchli,et al.  The control toolbox — An open-source C++ library for robotics, optimal and model predictive control , 2018, 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR).

[28]  Lorenzo Fagiano,et al.  Fast Nonlinear Model Predictive Control Using Set Membership Approximation⋆ ⋆This work has been partly supported by Ministero dell'Università e della Ricerca of Italy, under the National Project “Advanced control and identification techniques for innovative applications”. , 2008 .

[29]  Vijay Kumar,et al.  Approximating Explicit Model Predictive Control Using Constrained Neural Networks , 2018, 2018 Annual American Control Conference (ACC).

[30]  Tobias Weber,et al.  Implementation of Nonlinear Model Predictive Path-Following Control for an Industrial Robot , 2015, IEEE Transactions on Control Systems Technology.

[31]  Tim Jackson,et al.  Missing carbon reductions? Exploring rebound and backfire effects in UK households , 2010, Energy Policy.

[32]  Bernhard Schölkopf,et al.  Statistical Learning Theory: Models, Concepts, and Results , 2008, Inductive Logic.

[33]  D. Limón,et al.  Input-to-state stable MPC for constrained discrete-time nonlinear systems with bounded additive uncertainties , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[34]  M. Kothare,et al.  Efficient scheduled stabilizing model predictive control for constrained nonlinear systems , 2003 .

[35]  Antonio Ferramosca,et al.  Nonlinear MPC for Tracking Piece-Wise Constant Reference Signals , 2018, IEEE Transactions on Automatic Control.

[36]  H. Michalska,et al.  Receding horizon control of nonlinear systems , 1988, Proceedings of the 28th IEEE Conference on Decision and Control,.

[37]  Maxime Gautier,et al.  Dynamic identification of the Kuka LWR robot using motor torques and joint torque sensors data , 2014 .

[38]  Jan Swevers,et al.  Real-time nonlinear MPC and MHE for a large-scale mechatronic application , 2015 .

[39]  Rolf Findeisen,et al.  Modeling, parameter identification and model-based control of a lightweight robotic manipulator , 2013, 2013 IEEE International Conference on Control Applications (CCA).

[40]  Francesco Borrelli,et al.  An auto-generated nonlinear MPC algorithm for real-time obstacle avoidance of ground vehicles , 2013, 2013 European Control Conference (ECC).

[41]  Franco Blanchini,et al.  Set invariance in control , 1999, Autom..

[42]  R. E. Kalman,et al.  Contributions to the Theory of Optimal Control , 1960 .

[43]  Sergey Levine,et al.  Guided Policy Search via Approximate Mirror Descent , 2016, NIPS.

[44]  Lennart Ljung,et al.  Nonlinear System Identification: A User-Oriented Road Map , 2019, IEEE Control Systems.

[45]  Luigi Chisci,et al.  Systems with persistent disturbances: predictive control with restricted constraints , 2001, Autom..

[46]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[47]  Stephen P. Boyd,et al.  Fast Model Predictive Control Using Online Optimization , 2010, IEEE Transactions on Control Systems Technology.

[48]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[49]  David Q. Mayne,et al.  Tube‐based robust nonlinear model predictive control , 2011 .

[50]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[51]  Frank Allgöwer,et al.  A novel constraint tightening approach for nonlinear robust model predictive control , 2018, 2018 Annual American Control Conference (ACC).

[52]  Seungjoon Lee,et al.  Some manifold learning considerations toward explicit model predictive control , 2018, AIChE Journal.

[53]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[54]  Xiaojing Zhang,et al.  Optimization-Based Collision Avoidance , 2017, IEEE Transactions on Control Systems Technology.

[55]  Moritz Diehl,et al.  CasADi: a software framework for nonlinear optimization and optimal control , 2018, Mathematical Programming Computation.

[56]  Francesco Borrelli,et al.  Safe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks , 2019, 2019 American Control Conference (ACC).

[57]  Eduardo F. Camacho,et al.  MPC for tracking piecewise constant references for constrained linear systems , 2008, Autom..

[58]  Frank Allgöwer,et al.  Learning an Approximate Model Predictive Controller With Guarantees , 2018, IEEE Control Systems Letters.

[59]  Johannes Kohler,et al.  A nonlinear tracking model predictive control scheme for dynamic target signals , 2019, Automatica.

[60]  Johannes Köhler Distributed Economic Model Predictive Control under inexact minimization with application to power systems , 2017 .

[61]  P. Berkhout,et al.  Defining the rebound effect , 2000 .

[62]  Frank Allgöwer,et al.  Linear robust adaptive model predictive control: Computational complexity and conservatism , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[63]  Jan Tommy Gravdahl,et al.  On model predictive path following and trajectory tracking for industrial robots , 2017, 2017 13th IEEE Conference on Automation Science and Engineering (CASE).

[64]  Insup Lee,et al.  Verisig: verifying safety properties of hybrid systems with neural network controllers , 2018, HSCC.

[65]  Alessandro De Luca,et al.  Identifying the dynamic model used by the KUKA LWR: A reverse engineering approach , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[66]  H. ChenT,et al.  A Quasi-Infinite Horizon Nonlinear Model Predictive Control Scheme with Guaranteed Stability * , 1998 .

[67]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[68]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[69]  Frank Allgöwer,et al.  Safe and Fast Tracking on a Robot Manipulator: Robust MPC and Neural Network Control , 2020, IEEE Robotics and Automation Letters.

[70]  Mayuresh V. Kothare,et al.  An e!cient o"-line formulation of robust model predictive control using linear matrix inequalities (cid:1) , 2003 .

[71]  S. Sorrell,et al.  The rebound effect: Microeconomic definitions, limitations and extensions , 2008 .