The Influence of Speed and Position in Dynamic Gesture Recognition for Human-Robot Interaction

Human communication relies on several aspects beyond the speech. One of them is gestures as they express intentions, interests, feelings, or ideas and complement the speech. Social robots need to interpret these messages to allow a more natural Human-Robot Interaction. In this sense, our aim is to study the effect of position and speed features in dynamic gesture recognition. We use 3D information to extract the user’s skeleton and calculate the normalized position for all of its joints, and using the temporal variation of such positions, we calculate their speeds. Our three datasets are composed of 1355 samples from 30 users. We consider 14 common gestures in HRI involving upper body movements. A set of classification techniques is evaluated testing these three datasets to find what features perform better. Results indicate that the union of both speed and position achieves the best results among the three possibilities, 0.999 of - score. The combination that performs better to detect dynamic gestures in real time is finally integrated in our social robot with a simple HRI application to run a proof of concept test to check how the proposal behaves in a realistic scenario.

[1]  L. Breiman.  The Little Bootstrap and other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error , 1992 .

[2]  Davide Carneiro,et al.  A multi-modal approach for activity classification and fall detection , 2014, Int. J. Syst. Sci..

[3]  João Sequeira,et al.  A Multimodal Emotion Detection System during Human-Robot Interaction , 2013, Sensors.

[4]  María Malfaz,et al.  User Localization During Human-Robot Interaction , 2012, Sensors.

[5]  Fernando Alonso-Martín,et al.  INTEGRATION OF A VOICE RECOGNITION SYSTEM IN A SOCIAL ROBOT , 2011, Cybern. Syst..

[6]  María Malfaz,et al.  Vision-based people detection using depth information for social robots , 2017 .

[7]  Antonio Gentile,et al.  Modular Middleware for Gestural Data and Devices Management , 2017, J. Sensors.

[8]  Simon Fong,et al.  Gesture Recognition from Data Streams of Human Motion Sensor Using Accelerated PSO Swarm Search Feature Selection Algorithm , 2015, J. Sensors.

[9]  Diego G. S. Santos,et al.  HAGR-D: A Novel Approach for Gesture Recognition with Depth Maps , 2015, Sensors.

[10]  Taghi M. Khoshgoftaar,et al.  Deep learning applications and challenges in big data analytics , 2015, Journal of Big Data.

[11]  M. Alibali,et al.  Gesture's role in speaking, learning, and creating language. , 2013, Annual review of psychology.

[12]  José Carlos Castillo,et al.  Detecting and Classifying Human Touches in a Social Robot Through Acoustic Sensing and Machine Learning , 2017, Sensors.

[13]  Senén Barro,et al.  Do we need hundreds of classifiers to solve real world classification problems? , 2014, J. Mach. Learn. Res..

[14]  Sebastian Thrun,et al.  A Gesture Based Interface for Human-Robot Interaction , 2000, Auton. Robots.

[15]  Diane J. Cook,et al.  Human Activity Recognition and Pattern Discovery , 2010, IEEE Pervasive Computing.

[16]  María Malfaz,et al.  Study of Scenarios and Technical Requirements of a Social Assistive Robot for Alzheimer’s Disease Patients and Their Caregivers , 2015, International Journal of Social Robotics.

[17]  Antonio Fernández-Caballero,et al.  INT3-Horus framework for multispectrum activity interpretation in intelligent environments , 2013, Expert Syst. Appl..

[18]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[19]  Jonathan Tompson,et al.  MoDeep: A Deep Learning Framework Using Motion Features for Human Pose Estimation , 2014, ACCV.

[20]  Ruzena Bajcsy,et al.  Sequence of the Most Informative Joints (SMIJ): A new representation for human skeletal action recognition , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[21]  In-So Kweon,et al.  Upper Body Gesture Recognition for Human-Robot Interaction , 2011, HCI.

[22]  Antonio Fernández-Caballero,et al.  Human activity monitoring by local and global finite state machines , 2012, Expert Syst. Appl..

[23]  Guodong Guo,et al.  Evaluating spatiotemporal interest point features for depth-based action recognition , 2014, Image Vis. Comput..