Design and Implementation of Humanoid Robot Behavior Imitation System Based on Skeleton Tracking

This paper builds and implements a robot humanoid motion imitation system, which consists of three parts. Firstly, a Microsoft Kinect 2.0 somatosensory camera is applied to capture human skeleton data. Secondly, using the EWMA (exponentially weighted moving averages, EWMA), we can smooth the acquired skeleton data considering the local and entire process without prior knowledge of noise. And the Holt-Winters second-order EWMA is employed to track the original signal slope and incorporate its trend into the smoothed data. Then, the smoothed data are mapped to the robot joint angles by geometric calculations. Finally, a NAO robot implements motion imitation based on the joint angles. Experimental results demonstrate that the NAO robot can imitate the action of the instructor smoothly and steadily using the proposed motion imitation system.

[1]  Ales Ude,et al.  Enabling real-time full-body imitation: a natural way of transferring human movement to humanoids , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[2]  Ayşegül Uçar,et al.  Gesture imitation and recognition using Kinect sensor and extreme learning machines , 2016 .

[3]  Majid Nili Ahmadabadi,et al.  Inverse Kinematics Based Human Mimicking System using Skeletal Tracking Technology , 2017, J. Intell. Robotic Syst..

[4]  Changyin Sun,et al.  Adaptive Neural Impedance Control of a Robotic Manipulator With Input Saturation , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  Marcelo R. Campo,et al.  Easy gesture recognition for Kinect , 2014, Adv. Eng. Softw..

[6]  Andrea Lockerd Thomaz,et al.  Effects of nonverbal communication on efficiency and robustness in human-robot teamwork , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  F. Duan,et al.  A study of the human-robot synchronous control system based on skeletal tracking technology , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[8]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Ales Ude,et al.  Automatic Generation of Kinematic Models for the Conversion of Human Motion Capture Data into Humanoid Robot Motion , 2000 .

[10]  Radu Horaud,et al.  Hand Motion from 3D Point Trajectories and a Smooth Surface Model , 2004, ECCV.

[11]  Björn Krüger,et al.  A Study On Perceptual Similarity of Human Motions , 2011, VRIPHYS.

[12]  Malik Mallem,et al.  Real-time tele-operation and tele-walking of humanoid Robot Nao using Kinect Depth Camera , 2013, 2013 10th IEEE INTERNATIONAL CONFERENCE ON NETWORKING, SENSING AND CONTROL (ICNSC).

[13]  Katsu Yamane,et al.  Simultaneous tracking and balancing of humanoid robots for imitating human motion capture data , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Kerstin Dautenhahn,et al.  Socially intelligent robots: dimensions of human–robot interaction , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[15]  M. Mori.  THE UNCANNY VALLEY , 2020, The Monster Theory Reader.

[16]  Yangsheng Xu,et al.  A real-time human imitation system , 2012, Proceedings of the 10th World Congress on Intelligent Control and Automation.

[17]  Juan Manuel Ibarra Zannatha,et al.  Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation , 2013, Comput. Methods Programs Biomed..

[18]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.