Vision-Based Hand Gesture Recognition for Human-Robot Collaboration: A Survey

Recently, human-robot collaboration has attracted many research interests. Unlike traditional industrial robots, human-robot collaboration combines the advantages of robots with the flexibility and cognitive skills of human worker to work together in a shared manufacturing environment. However, to achieve an efficient human-robot collaboration, the communication channels between human workers and robots needs to be tackled. Vision-based hand gesture recognition has been effectively applied as the interface between humans and computer for long time. So, this paper presents a survey of the vision-based hand gesture recognition research in human-robot collaborative manufacturing. We first propose an overall model of vision-based hand gesture recognition for human-robot collaboration. Then, we review three essential technical components of this model: sensor technologies, hand gesture detection and segmentation and hand gesture classification. In addition, the relevant techniques for vision-based hand gesture recognition are compared. Towards the end of this paper, future research trends are discussed.

[1]  Francesco Leali,et al.  Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications , 2018, Mechatronics.

[2]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[3]  Frank Weichert,et al.  Analysis of the Accuracy and Robustness of the Leap Motion Controller , 2013, Sensors.

[4]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[5]  Tony Belpaeme,et al.  Child Speech Recognition in Human-Robot Interaction: Evaluations and Recommendations , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[6]  Mariusz Oszust,et al.  Recognition of signed dynamic expressions observed by ToF camera , 2013, 2013 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA).

[7]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[8]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[9]  Ai Poh Loh,et al.  Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds , 2012, International Journal of Computer Vision.

[10]  Pong C. Yuen,et al.  Multi-cue Visual Tracking Using Robust Feature-Level Fusion Based on Joint Sparse Representation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Debi Prosad Dogra,et al.  A novel framework of continuous human-activity recognition using Kinect , 2018, Neurocomputing.

[13]  Holger Regenbrecht,et al.  A leap-supported, hybrid AR interface approach , 2013, OZCHI.

[14]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  F. Lucht,et al.  Implication of a Healthcare Worker With Chronic Skin Disease in the Transmission of an Epidemic Strain of Methicillin-Resistant Staphylococcus aureus in a Pediatric Intensive Care Unit , 2003, Infection Control & Hospital Epidemiology.

[16]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[17]  Robin R. Murphy,et al.  Hand gesture recognition with depth images: A review , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[18]  Joze Guna,et al.  An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking , 2014, Sensors.

[19]  Haitham Badi.  Retraction Note to: A Survey on Recent Vision-Based Gesture Recognition , 2017 .

[20]  Robert X. Gao,et al.  Cloud-enabled prognosis for manufacturing , 2015 .

[21]  Václav Hlavác,et al.  Pose primitive based human action recognition in videos or still images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Sotiris Makris,et al.  Human–robot interaction review and challenges on task planning and programming , 2016, Int. J. Comput. Integr. Manuf..

[23]  Lu Yang,et al.  Survey on 3D Hand Gesture Recognition , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Honghai Liu,et al.  Gesture Recognition Based on Kinect and sEMG Signal Fusion , 2018, Mobile Networks and Applications.

[25]  Chong Wang,et al.  Superpixel-Based Hand Gesture Recognition With Kinect Depth Camera , 2015, IEEE Transactions on Multimedia.

[26]  Masatoshi Ishikawa,et al.  High-speed Human / Robot Hand Interaction System , 2015, HRI.

[27]  Rodina Binti Ahmad,et al.  A systematic literature review on vision based gesture recognition techniques , 2018, Multimedia Tools and Applications.

[28]  Haitham Badi,et al.  RETRACTED ARTICLE: A Survey on Recent Vision-Based Gesture Recognition , 2016 .

[29]  Sergio Escalera,et al.  A Survey on Deep Learning Based Approaches for Action and Gesture Recognition in Image Sequences , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[30]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[31]  Sebastian van Delden,et al.  Pick-and-place application development using voice and visual commands , 2012, Ind. Robot.

[32]  Simone Calderara,et al.  Visual Tracking: An Experimental Survey , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Lihui Wang,et al.  Gesture recognition for human-robot collaboration: A review , 2017, International Journal of Industrial Ergonomics.

[34]  Marcus A. Magnor,et al.  A Survey on Time-of-Flight Stereo Fusion , 2013, Time-of-Flight and Depth Imaging.

[35]  Shuzhi Sam Ge,et al.  3D pointing gesture recognition for human-robot interaction , 2016, 2016 Chinese Control and Decision Conference (CCDC).

[36]  Hui Deng,et al.  A Survey on Hand Gesture Recognition , 2013, 2013 International Conference on Computer Sciences and Applications.

[37]  D. Jude Hemanth,et al.  Hand posture and gesture recognition techniques for virtual reality applications: a survey , 2016, Virtual Reality.

[38]  Jörg Stückler,et al.  Learning to interpret pointing gestures with a time-of-flight camera , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[39]  Martin Buss,et al.  Human-Robot Collaboration: a Survey , 2008, Int. J. Humanoid Robotics.