Bridging the Model-Reality Gap With Lipschitz Network Adaptation

As robots venture into the real world, they are subject to unmodeled dynamics and disturbances. Traditional model-based control approaches have been proven successful in relatively static and known operating environments. However, when an accurate model of the robot is not available, model-based design can lead to suboptimal and even unsafe behaviour. In this work, we propose a method that bridges the model-reality gap and enables the application of model-based approaches even if dynamic uncertainties are present. In particular, we present a learning-based model reference adaptation approach that makes a robot system, with possibly uncertain dynamics, behave as a predefined reference model. In turn, the reference model can be used for model-based controller design. In contrast to typical model reference adaptation control approaches, we leverage the representative power of neural networks to capture highly nonlinear dynamics uncertainties and guarantee stability by encoding a certifying Lipschitz condition in the architectural design of a special type of neural network called the Lipschitz network. Our approach applies to a general class of nonlinear control-affine systems even when our prior knowledge about the true robot system is limited. We demonstrate our approach in flying inverted pendulum experiments, where an off-the-shelf quadrotor is challenged to balance an inverted pendulum while hovering or tracking circular trajectories.

[1]  Angela P. Schoellig,et al.  Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning , 2021, Annu. Rev. Control. Robotics Auton. Syst..

[2]  Chelsea Finn,et al.  Deep Reinforcement Learning amidst Lifelong Non-Stationarity , 2020, ArXiv.

[3]  Markus Hehn,et al.  A flying inverted pendulum , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Angela P. Schoellig,et al.  Learning‐based Nonlinear Model Predictive Control to Improve Vision‐based Mobile Robot Path Tracking , 2016, J. Field Robotics.

[5]  Matthew R. James,et al.  Numerical approximation of the H∞ norm for nonlinear systems , 1995, Autom..

[6]  Gang Tao,et al.  Robust Backstepping Sliding-Mode Control and Observer-Based Fault Estimation for a Quadrotor UAV , 2016, IEEE Transactions on Industrial Electronics.

[7]  Angela P. Schoellig,et al.  Deep neural networks as add-on modules for enhancing robot performance in impromptu trajectory tracking , 2020, Int. J. Robotics Res..

[8]  Hassan K. Khalil,et al.  Adaptive control of a class of nonlinear discrete-time systems using neural networks , 1995, IEEE Trans. Autom. Control..

[9]  R. Sarpong,et al.  Bio-inspired synthesis of xishacorenes A, B, and C, and a new congener from fuscol† †Electronic supplementary information (ESI) available. See DOI: 10.1039/c9sc02572c , 2019, Chemical science.

[10]  Zexiang Li,et al.  Disturbance Observer Based Hovering Control of Quadrotor Tail-Sitter VTOL UAVs Using H∞ Synthesis , 2018, IEEE Robotics Autom. Lett..

[11]  Angela P. Schoellig,et al.  Experience Selection Using Dynamics Similarity for Efficient Multi-Source Transfer Learning Between Robots , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Soon-Jo Chung,et al.  Neural Lander: Stable Drone Landing Control Using Learned Dynamics , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[13]  Chengyu Cao,et al.  The use of learning in fast adaptation algorithms , 2014 .

[14]  Erdal Kayacan,et al.  Knowledge Transfer Between Robots with Similar Dynamics for High-Accuracy Impromptu Trajectory Tracking , 2019, 2019 18th European Control Conference (ECC).

[15]  Dominique Bonvin,et al.  Data-driven estimation of the infinity norm of a dynamical system , 2007, 2007 46th IEEE Conference on Decision and Control.

[16]  J. Doyle,et al.  Essentials of Robust Control , 1997 .

[17]  W. Marsden I and J , 2012 .

[18]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[19]  Eduardo F. Camacho,et al.  Robust tube-based MPC for tracking of constrained linear systems with additive disturbances , 2010 .

[20]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[21]  Christopher D. McKinnon,et al.  Learn Fast, Forget Slow: Safe Predictive Learning Control for Systems With Unknown and Changing Dynamics Performing Repetitive Tasks , 2018, IEEE Robotics and Automation Letters.

[22]  Hassan K. Khalil,et al.  Nonlinear Systems Third Edition , 2008 .

[23]  Naira Hovakimyan,et al.  L1 Adaptive Control Theory - Guaranteed Robustness with Fast Adaptation , 2010, Advances in design and control.

[24]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Cem Anil,et al.  Sorting out Lipschitz function approximation , 2018, ICML.

[26]  Manfred Morari,et al.  Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks , 2019, NeurIPS.

[27]  George W. Irwin,et al.  Direct neural model reference adaptive control , 1995 .

[28]  Girish Chowdhary,et al.  Deep Model Reference Adaptive Control , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[29]  Jonathan P. How,et al.  Bayesian Nonparametric Adaptive Control Using Gaussian Processes , 2015, IEEE Transactions on Neural Networks and Learning Systems.