Rich and robust human-robot interaction on gesture recognition for assembly tasks

The adoption of robotics technology has the potential to advance quality, efficiency and safety for manufacturing enterprises, in particular small and medium-sized enterprises. This paper presents a human-robot interaction (HRI) system that enables a robot to receive commands, provide information to a human teammate and ask them a favor. In order to build a robust HRI system based on gesture recognition, three key issues are addressed: richness, multiple feature fusion and failure verification. The developed system has been tested and validated in a realistic lab with a real mobile manipulator and a human teammate to solve a puzzle game.

[1]  Anders Robertsson,et al.  On Distributed Knowledge Bases for Robotized Small-Batch Assembly , 2015, IEEE Transactions on Automation Science and Engineering.

[2]  Jörg Stückler,et al.  Real-time object detection, localization and verification for fast robotic depalletizing , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Mohamed Chetouani,et al.  Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement , 2014, Front. Neurorobot..

[4]  Gi Hyun Lim,et al.  3D object perception and perceptual learning in the RACE project , 2016, Robotics Auton. Syst..

[5]  Yong Luo,et al.  Group Sparse Multiview Patch Alignment Framework With View Consistency for Image Classification , 2014, IEEE Transactions on Image Processing.

[6]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[7]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Rainer Bischoff,et al.  The Strategic Research Agenda for Robotics in Europe [Industrial Activities] , 2010 .

[9]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[10]  Il Hong Suh,et al.  Robust robot knowledge instantiation for intelligent service robots , 2010, Intell. Serv. Robotics.

[11]  Rainer Bischoff,et al.  Robotic Visions to 2020 and beyond -- The strategic research agenda for robotics in Europe , 2009 .

[12]  Martial Hebert,et al.  Unsupervised Learning using Sequential Verification for Action Recognition , 2016, ArXiv.

[13]  Bernardo Cunha,et al.  A Skill-Based Architecture for Pick and Place Manipulation Tasks , 2015, EPIA.

[14]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[15]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[16]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[17]  Giovanni Pezzulo,et al.  The “Interaction Engine”: A Common Pragmatic Competence Across Linguistic and Nonlinguistic Interactions , 2012, IEEE Transactions on Autonomous Mental Development.

[18]  Trevor Darrell,et al.  Fast concurrent object localization and recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Bruno Siciliano,et al.  EuRoC - The Challenge Initiative for European Robotics , 2014, ISR 2014.

[20]  Luís Paulo Reis,et al.  Skill-based anytime agent architecture for logistics and manipulation tasks: EuRoC Challenge 2, Stage II - Realistic Labs: Benchmarking , 2017, 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[21]  Yi Li,et al.  Hand gesture recognition using Kinect , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering.

[22]  Sukhendu Das,et al.  A Survey of Decision Fusion and Feature Fusion Strategies for Pattern Classification , 2010, IETE Technical Review.

[23]  Jian Yang,et al.  Feature fusion: parallel strategy vs. serial strategy , 2003, Pattern Recognit..

[24]  Peter Robinson,et al.  Cooperative gestures: Effective signaling for humanoid robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[25]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[26]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[27]  Sergio Escalera,et al.  A real-time Human-Robot Interaction system based on gestures for assistive scenarios , 2016, Comput. Vis. Image Underst..

[28]  Karon E. MacLean,et al.  Gestures for industry Intuitive human-robot communication from human observation , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).