Knowledge Transfer Between Robots with Similar Dynamics for High-Accuracy Impromptu Trajectory Tracking

In this paper, we propose an online learning approach that enables the inverse dynamics model learned for a source robot to be transferred to a target robot (e.g., from one quadrotor to another quadrotor with different mass or aerodynamic properties). The goal is to leverage knowledge from the source robot such that the target robot achieves high-accuracy trajectory tracking on arbitrary trajectories from the first attempt with minimal data recollection and training. Most existing approaches for multi-robot knowledge transfer are based on post-analysis of datasets collected from both robots. In this work, we study the feasibility of impromptu transfer of models across robots by learning an error prediction module online. In particular, we analytically derive the form of the mapping to be learned by the online module for exact tracking, propose an approach for characterizing similarity between robots, and use these results to analyze the stability of the overall system. The proposed approach is illustrated in simulation and verified experimentally on two different quadrotors performing impromptu trajectory tracking tasks, where the quadrotors are required to accurately track arbitrary hand-drawn trajectories from the first attempt.

[1]  Martial Hebert,et al.  Learning Transferable Policies for Monocular Reactive MAV Control , 2016, ISER.

[2]  Sergey Levine,et al.  Learning modular neural network policies for multi-task and multi-robot transfer , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Jan Peters,et al.  Alignment-based transfer learning for robot models , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[4]  Angela P. Schoellig,et al.  Transfer learning for high‐precision trajectory tracking through L1 adaptive feedback and iterative learning , 2018, International Journal of Adaptive Control and Signal Processing.

[5]  Angela P. Schoellig,et al.  Data-Efficient Multirobot, Multitask Transfer Learning for Trajectory Tracking , 2017, IEEE Robotics and Automation Letters.

[6]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[7]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Angela P. Schoellig,et al.  Multi-robot transfer learning: A dynamical system perspective , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Eric Eaton,et al.  Unsupervised Cross-Domain Transfer in Policy Gradient Reinforcement Learning via Manifold Alignment , 2015, AAAI.

[10]  Sergey Levine,et al.  Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning , 2017, ICLR.

[11]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[12]  Raffaello D'Andrea,et al.  Knowledge transfer for high-performance quadrocopter maneuvers , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Alessandro Lazaric,et al.  Transfer in Reinforcement Learning: A Framework and a Survey , 2012, Reinforcement Learning.

[14]  Danwei W. Wang,et al.  Analysis of Nonlinear Discrete-Time Systems with Higher-Order Iterative Learning Control , 2001 .

[15]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[16]  Eduardo Sontag Input to State Stability: Basic Concepts and Results , 2008 .

[17]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[18]  Angela P. Schoellig,et al.  Distributed iterative learning control for a team of quadrotors , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[19]  Sergey Levine,et al.  One-Shot Visual Imitation Learning via Meta-Learning , 2017, CoRL.

[20]  Changshui Zhang,et al.  Transferred Dimensionality Reduction , 2008, ECML/PKDD.

[21]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[22]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[23]  Angela P. Schoellig,et al.  Design of deep neural networks as add-on blocks for improving impromptu trajectory tracking , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[24]  Robert Hall,et al.  Similarity Metrics for Closed-Loop Dynamic Systems , 2008 .

[25]  Chong-Ho Choi,et al.  Iterative learning control for discrete-time nonlinear systems , 1994 .

[26]  Angela P. Schoellig,et al.  Deep neural networks for improved, impromptu trajectory tracking of quadrotors , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Andreas Krause,et al.  Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Sergey Levine,et al.  One-shot learning of manipulation skills with online dynamics adaptation and neural network priors , 2015, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).