Spatial and rotation invariant 3D gesture recognition based on sparse representation

Advances in motion tracking technology, especially for commodity hardware, still require robust 3D gesture recognition in order to fully exploit the benefits of natural user interfaces. In this paper, we introduce a novel 3D gesture recognition algorithm based on the sparse representation of 3D human motion. The sparse representation of human motion provides a set of features that can be used to efficiently classify gestures in real-time. Compared to existing gesture recognition systems, sparse representation, the proposed approach enables full spatial and rotation invariance and provides high tolerance to noise. Moreover, the proposed classification scheme takes into account the inter-user variability which increases gesture classification accuracy in user-independent scenarios. We validated our approach with existing motion databases for gestural interaction and performed a user evaluation with naive subjects to show its robustness to arbitrarily defined gestures. The results showed that our classification scheme has high classification accuracy for user-independent scenarios even with users who have different handedness. We believe that sparse representation of human motion will pave the way for a new generation of 3D gesture recognition systems in order to fully open the potential of natural user interfaces.

[1]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[2]  Günter Hommel,et al.  Velocity Profile Based Recognition of Dynamic Gestures with Discrete Hidden Markov Models , 1997, Gesture Workshop.

[3]  Juan Pablo Wachs,et al.  Context-based hand gesture recognition for the operating room , 2014, Pattern Recognit. Lett..

[4]  Philippe Fuchs,et al.  User-defined gestural interaction: A study on gesture memorization , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[5]  Biing-Hwang Juang,et al.  6DMG: a new 6D motion gesture database , 2012, MMSys '12.

[6]  Biing-Hwang Juang,et al.  Feature Processing and Modeling for 6D Motion Gesture Recognition , 2013, IEEE Transactions on Multimedia.

[7]  Kongqiao Wang,et al.  Hand gesture recognition and virtual game control based on 3D accelerometer and EMG sensors , 2009, IUI.

[8]  Joseph J. LaViola,et al.  3D Gesture classification with linear acceleration and angular velocity sensing devices for video games , 2013, Entertain. Comput..

[9]  Michael Rohs,et al.  Protractor3D: a closed-form solution to rotation-invariant 3D gestures , 2011, IUI '11.

[10]  I. Scott MacKenzie,et al.  Accuracy measures for evaluating computer pointing devices , 2001, CHI.

[11]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  Radu-Daniel Vatavu,et al.  Gestures as point clouds: a $P recognizer for user interface prototypes , 2012, ICMI '12.

[14]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[15]  Marc Erich Latoschik A user interface framework for multimodal VR interactions , 2005, ICMI '05.

[16]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[17]  Timo Pylvänäinen,et al.  Accelerometer Based Gesture Recognition Using Continuous HMMs , 2005, IbPRIA.

[18]  L. R. Rabiner,et al.  A comparative study of several dynamic time-warping algorithms for connected-word recognition , 1981, The Bell System Technical Journal.

[19]  Janusz Konrad,et al.  A gesture-driven computer interface using Kinect , 2012, 2012 IEEE Southwest Symposium on Image Analysis and Interpretation.

[20]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[21]  Radu-Daniel Vatavu,et al.  The impact of motion dimensionality and bit cardinality on the design of 3D gesture recognizers , 2013, Int. J. Hum. Comput. Stud..

[22]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[23]  Joseph J. LaViola,et al.  Breaking the status quo: Improving 3D gesture recognition with spatially convenient input devices , 2010, 2010 IEEE Virtual Reality Conference (VR).

[24]  Jani Mäntyjärvi,et al.  Accelerometer-based gesture control for a design environment , 2006, Personal and Ubiquitous Computing.

[25]  Panagiotis Tsakalides,et al.  Sparse representations for hand gesture recognition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Zhigang Deng,et al.  Quaternion space sparse decomposition for motion compression and retrieval , 2012, SCA '12.

[27]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[28]  Fabien Moutarde,et al.  Gesture Recognition Using a Depth Camera for Human Robot Collaboration on Assembly Line , 2015 .

[29]  Yi Li,et al.  Learning shift-invariant sparse representation of actions , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[31]  Joseph J. LaViola,et al.  3D Gestural Interaction: The State of the Field , 2013 .

[32]  Giorgio Metta,et al.  Keep it simple and sparse: real-time action recognition , 2013, J. Mach. Learn. Res..

[33]  Yael Edan,et al.  Technical Brief: A Gesture-based Tool for Sterile Browsing of Radiology Images , 2008, J. Am. Medical Informatics Assoc..

[34]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Tuukka M. Takala,et al.  Survey of 3DUI applications and development challenges , 2012, 2012 IEEE Symposium on 3D User Interfaces (3DUI).

[36]  Tadashi Yoshidome,et al.  Cooperative welfare robot system using hand gesture instructions , 2004 .

[37]  Quentin Barthelemy,et al.  Decomposition and dictionary learning for 3D trajectories , 2014, Signal Process..

[38]  Zhen Wang,et al.  uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications , 2009, PerCom.

[39]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[40]  Yang Li,et al.  Protractor: a fast and accurate gesture recognizer , 2010, CHI.

[41]  Radu-Daniel Vatavu,et al.  The effect of sampling rate on the performance of template-based gesture recognizers , 2011, ICMI '11.