Robot teaching by teleoperation based on visual interaction and extreme learning machine

Abstract Compared with traditional robot teaching methods, robots can learn various human-like skills in a more efficient and natural manner through teleoperation. In this paper, we propose a teleoperation method based on human-robot interaction (HRI), which mainly uses visual information. With only one teleoperation, the robot can reproduce a trajectory. There is a certain error between this trajectory and the optimal trajectory due to the cause of the human demonstrator or the robot. So we use an extreme learning machine (ELM) based algorithm to transfer the demonstrator’s motions to the robot. To verify the method, we use a Microsoft KinectV2 to capture the human body motion and the hand state, according to which a Baxter robot in Virtual Robot Experimentation Platform (V-REP) will be controlled by the command. Through learning and training by the ELM, the robot in V-REP can complete a certain task autonomously and the robot in reality can reproduce this trajectory well. The experimental results show that the developed method has achieved satisfactory performance.

[1]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[2]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[3]  Guido M. Cortelazzo,et al.  Time-of-Flight Cameras and Microsoft Kinect™ , 2012, Springer Briefs in Electrical and Computer Engineering.

[4]  Thomas B. Sheridan,et al.  Human–Robot Interaction , 2016, Hum. Factors.

[5]  Antonio Bicchi,et al.  Teaching by demonstration on dual-arm robot using variable stiffness transferring , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[6]  Chenguang Yang,et al.  Human–Robot Interaction Interface , 2016 .

[7]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[8]  Surya P. N. Singh,et al.  V-REP: A versatile and scalable robot simulation framework , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Lijun Zhao,et al.  Intuitive robot teaching by hand guided demonstration , 2016, 2016 IEEE International Conference on Mechatronics and Automation.

[10]  Tadej Petric,et al.  Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach , 2014, Auton. Robots.

[11]  Z. Li,et al.  Teleoperation control of Baxter robot using body motion tracking , 2014, 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI).

[12]  Nobuto Matsuhira,et al.  Virtual Robot Experimentation Platform V-REP: A Versatile 3D Robot Simulator , 2010, SIMPAR.

[13]  Simone Zennaro.  Evaluation of Microsoft Kinect 360 and Microsoft Kinect One for robotics and computer vision applications , 2014 .

[14]  Raymond W. McGorry,et al.  The validity of the first and second generation Microsoft Kinect™ for identifying joint center locations during static postures. , 2015, Applied ergonomics.

[15]  Wassim Jabi,et al.  A Collaborative Approach to Digital Fabrication: A Case Study for the Design and Production of Concrete ‘Pop-up’ Structures , 2015 .

[16]  Vincent Padois,et al.  Tools for dynamics simulation of robots: a survey based on user feedback , 2014, ArXiv.

[17]  Darwin G. Caldwell,et al.  Human-robot skills transfer interfaces for a flexible surgical robot , 2014, Comput. Methods Programs Biomed..

[18]  Lin Yang,et al.  Evaluating and Improving the Depth Accuracy of Kinect for Windows v2 , 2015, IEEE Sensors Journal.

[19]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[20]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[21]  Joseph J. LaViola,et al.  Natural User Interfaces for Adjustable Autonomy in Robot Control , 2015, IEEE Computer Graphics and Applications.

[22]  Sheng Liu,et al.  Teaching and learning of deburring robots using neural networks , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[23]  Klaus Neumann,et al.  Neural learning of stable dynamical systems based on data-driven Lyapunov candidates , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Taewoo Kim,et al.  A robot teaching framework for a redundant dual arm manipulator with teleoperation from exoskeleton motion data , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.