Survey of imitation learning for robotic manipulation

With the development of robotics, the application of robots has gradually evolved from industrial scenes to more intelligent service scenarios. For multitasking operations of robots in complex and uncertain environments, the traditional manual coding method is not only cumbersome but also unable to adapt to sudden changes in the environment. Imitation learning that avoids learning skills from scratch by using the expert demonstration has become the most effective way for robotic manipulation. The paper is intended to provide the survey of imitation learning of robotic manipulation and explore the future research trend. The review of the art of imitation learning for robotic manipulation involves three aspects that are demonstration, representation and learning algorithms. Towards the end of the paper, we highlight areas of future research potential.

[1]  Kensuke Harada,et al.  Teaching Robots to Do Object Assembly using Multi-modal 3D Vision , 2016, Neurocomputing.

[2]  Pieter Abbeel,et al.  An Algorithmic Perspective on Imitation Learning , 2018, Found. Trends Robotics.

[3]  François Charpillet,et al.  Multi-modal Intention Prediction with Probabilistic Movement Primitives , 2017, HFR.

[4]  Majid Nili Ahmadabadi,et al.  A Fast, Robust, and Incremental Model for Learning High-Level Concepts From Human Motions by Imitation , 2017, IEEE Transactions on Robotics.

[5]  J. Andrew Bagnell,et al.  Maximum margin planning , 2006, ICML.

[6]  Aude Billard,et al.  A survey of Tactile Human-Robot Interactions , 2010, Robotics Auton. Syst..

[7]  Jin Zhao,et al.  Motion mapping of the heterogeneous master–slave system for intuitive telemanipulation , 2018 .

[8]  J A Bagnell,et al.  An Invitation to Imitation , 2015 .

[9]  Yi Guo,et al.  Learning How Pedestrians Navigate: A Deep Inverse Reinforcement Learning Approach , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Chenguang Yang,et al.  Neural-Learning-Based Telerobot Control With Guaranteed Performance , 2017, IEEE Transactions on Cybernetics.

[11]  Silvio Savarese,et al.  ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation , 2018, CoRL.

[12]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[13]  Ken Goldberg,et al.  Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation , 2017, ICRA.

[14]  Sergey Levine,et al.  Time-Contrastive Networks: Self-Supervised Learning from Video , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[16]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[17]  Joelle Pineau,et al.  OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning , 2017, AAAI.

[18]  E. AmirM.Ghalamzan,et al.  Robot learning from demonstrations: Emulation learning in environments with moving obstacles , 2018, Robotics Auton. Syst..

[19]  Jianwei Zhang,et al.  Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task , 2016, CAAI Trans. Intell. Technol..

[20]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[21]  Antonio Bicchi,et al.  Development of a robotic teaching interface for human to human skill transfer , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  G. Schreiber,et al.  The Fast Research Interface for the KUKA Lightweight Robot , 2022 .

[23]  Mansour A. Karkoub,et al.  Humanoid Robot's Visual Imitation of 3-D Motion of a Human Subject Using Neural-Network-Based Inverse Kinematics , 2016, IEEE Systems Journal.

[24]  Andrej Gams,et al.  Coupling Movement Primitives: Interaction With the Environment and Bimanual Tasks , 2014, IEEE Transactions on Robotics.

[25]  Shie Mannor,et al.  End-to-End Differentiable Adversarial Imitation Learning , 2017, ICML.

[26]  Matthieu Geist,et al.  Bridging the Gap Between Imitation Learning and Inverse Reinforcement Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Feng Gao,et al.  Feeling the force: Integrating force and pose for fluent discovery through imitation learning to open medicine bottles , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Cheng Fang,et al.  A DMPs-Based Framework for Robot Learning and Generalization of Humanlike Variable Impedance Skills , 2018, IEEE/ASME Transactions on Mechatronics.

[29]  Ruifeng Li,et al.  Interface Design of a Physical Human–Robot Interaction System for Human Impedance Adaptive Skill Transfer , 2018, IEEE Transactions on Automation Science and Engineering.

[30]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[31]  Mohamed Medhat Gaber,et al.  Imitation Learning , 2017, ACM Comput. Surv..

[32]  Oliver Kroemer,et al.  Interaction primitives for human-robot cooperation tasks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Benjamin Rosman,et al.  Transfer Learning for Prosthetics Using Imitation Learning , 2019, ArXiv.

[34]  Yongxin Chen,et al.  On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator , 2019, ArXiv.

[35]  Sergey Levine,et al.  Learning Dexterous Manipulation Policies from Experience and Imitation , 2016, ArXiv.

[36]  Sergey Levine,et al.  Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations , 2017, Robotics: Science and Systems.

[37]  Chenfanfu Jiang,et al.  Mirroring without Overimitation: Learning Functionally Equivalent Manipulation Actions , 2019, AAAI.

[38]  Sergey Levine,et al.  Learning Robust Rewards with Adversarial Inverse Reinforcement Learning , 2017, ICLR 2017.

[39]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[40]  Christopher G. Atkeson,et al.  Optimization and learning for rough terrain legged locomotion , 2011, Int. J. Robotics Res..

[41]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[42]  Masashi Sugiyama,et al.  Imitation Learning from Imperfect Demonstration , 2019, ICML.

[43]  Sergey Levine,et al.  Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization , 2016, ICML.

[44]  Jun Morimoto,et al.  Skill learning and action recognition by arc-length dynamic movement primitives , 2018, Robotics Auton. Syst..

[45]  Alexandre Attia,et al.  Global overview of Imitation Learning , 2018, ArXiv.

[46]  Di Guo,et al.  Robotic teleoperation systems using a wearable multimodal fusion device , 2017 .

[47]  Di Guo,et al.  Development of a Wearable Device for Motion Capturing Based on Magnetic and Inertial Measurement Units , 2017, Sci. Program..

[48]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[49]  Peter Stone,et al.  Behavioral Cloning from Observation , 2018, IJCAI.

[50]  Lihui Wang,et al.  Gesture recognition for human-robot collaboration: A review , 2017, International Journal of Industrial Ergonomics.