High Level Robot Programming Using Body and Hand Gestures

Abstract Robot programming software tools are expected to be more intuitive and user friendly. This paper proposes a method for simplifying industrial robot programming using visual sensors detecting the human motions. A vocabulary of body and hand gestures is defined, allowing the movement of robot in different directions. An external controller application is used for the transformation between human and robot motions. On the robot side, a decoder application is developed translating the human messages into robot motions. The method is integrated within an open communication architecture based on Robot Operating System (ROS), enabling thus the easy extensibility with new functionalities. An automotive industry case study demonstrated the method, including the commanding of a dual arm robot for single and bi-manual motions.

[1]  Carlos Morato,et al.  Toward Safe Human Robot Collaboration by Using Multiple Kinects Based Real-Time Human Tracking , 2014, J. Comput. Inf. Sci. Eng..

[2]  Tom Duckett,et al.  Position teaching of a robot arm by demonstration with a wearable input device , 2004 .

[3]  J. Gregory Trafton,et al.  Anticipation of Touch Gestures to Improve Robot Reaction Time , 2015, AAAI Fall Symposia.

[4]  Sotiris Makris,et al.  Human–robot interaction review and challenges on task planning and programming , 2016, Int. J. Comput. Integr. Manuf..

[5]  Guillem Alenyà,et al.  Teaching Grasping Points Using Natural Movements , 2015, CCIA.

[6]  Robin R. Murphy,et al.  Hand gesture recognition with depth images: A review , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[7]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[8]  Sotiris Makris,et al.  Robotized Assembly Process Using Dual Arm Robot , 2014 .

[9]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[10]  Junsong Yuan,et al.  Depth camera based hand gesture recognition and its applications in Human-Computer-Interaction , 2011, 2011 8th International Conference on Information, Communications & Signal Processing.

[11]  Sebastian Thrun,et al.  A Gesture Based Interface for Human-Robot Interaction , 2000, Auton. Robots.

[12]  Sotiris Makris,et al.  Intuitive dual arm robot programming for assembly operations , 2014 .

[13]  Alexander H. Waibel,et al.  Natural human-robot interaction using speech, head pose and gestures , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[14]  Pedro Neto,et al.  High-level programming and control for industrial robotics: using a hand-held accelerometer-based input device for gesture and posture recognition , 2010, Ind. Robot.

[15]  Geoffrey Biggs,et al.  A Survey of Robot Programming Systems , 2010 .

[16]  Seong-Whan Lee,et al.  Gesture Spotting and Recognition for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[17]  Kun Qian,et al.  Developing a Gesture Based Remote Human-Robot Interaction System Using Kinect , 2013 .

[18]  Maja J. Mataric,et al.  Automated Proxemic Feature Extraction and Behavior Recognition: Applications in Human-Robot Interaction , 2013, Int. J. Soc. Robotics.

[19]  Rainer Stiefelhagen,et al.  Visual recognition of pointing gestures for human-robot interaction , 2007, Image Vis. Comput..

[20]  Luc Van Gool,et al.  Real-time 3D hand gesture interaction with a robot for understanding directions from humans , 2011, 2011 RO-MAN.

[21]  Aude Billard,et al.  Stochastic gesture production and recognition model for a humanoid robot , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[22]  Sarosh Patel,et al.  Real-Time Robot Control Using Leap Motion A Concept of Human-Robot Interaction , 2015 .

[23]  J. Norberto Pires.  Robot-by-voice: experiments on commanding an industrial robot using the human voice , 2005, Ind. Robot.

[24]  Pedro Neto,et al.  Real-time and continuous hand gesture spotting: An approach based on artificial neural networks , 2013, 2013 IEEE International Conference on Robotics and Automation.