Phase synchronization in a manifold space for recognizing dynamic hand gestures from periodic image sequence

Phase synchronization issue, that is caused by spotting gestures from video stream, varying frame-rates, speed of subject's implementation, should be overcome in developing Human-Computer Interaction (HCI) application using dynamic hand gestures. This paper tackles an interpolation technique to efficiently solve this issue. We firstly propose a new representation of dynamic hand gestures space that consists of both spatial and temporal features extracted from the hand gestures. The spatial features are extracted based on a manifold learning technique (ISOMAP) that takes into account non-linear features (e.g., poses of hand, illumination conditions, hand-shape differences). The temporal features handle hand movements thanks to Kanade-Lucas-Tomasi (KLT), good feature points tracking algorithm. We then propose an efficient interpolation scheme on the constructed space of hand gestures. This scheme ensures inter-period phase continuity as well as normalizes length of the hand gestures. We examine the proposed method with three different large datasets of dynamic hand gestures. Evaluation results confirm that the best accuracy rate achieves at 98% that is significantly higher than results from previous works (at 94%). The proposed method suggests a feasible and robust solution addressing technical issues in developing HCI application using the hand gestures to control home appliance devices.

[1]  Michael Cohen,et al.  Enhancing and experiencing spacetime resolution with videos and stills , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[2]  Thanh-Hai Tran,et al.  A combination of user-guide scheme and kernel descriptor on RGB-D data for robust and realtime hand posture recognition , 2016, Eng. Appl. Artif. Intell..

[3]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[4]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[5]  Z. Liu,et al.  A real time system for dynamic hand gesture recognition with a depth sensor , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[6]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[7]  Thanh-Hai Tran,et al.  Recognition of hand gestures from cyclic hand movements using spatial-temporal features , 2015, SoICT.

[8]  Masahiko Yachida,et al.  A Fast Algorithm of Video Super-Resolution Using Dimensionality Reduction by DCT and Example Selection , 2008 .

[9]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Takahiro Okabe,et al.  Video Temporal Super-Resolution Based on Self-similarity , 2010, ACCV.

[11]  Katarzyna Barczewska,et al.  Comparison of methods for hand gesture recognition based on Dynamic Time Warping algorithm , 2013, 2013 Federated Conference on Computer Science and Information Systems.

[12]  Dandu Amarnatha Reddy.  Vision Based Hand Gesture Recognition for Human Computer Interaction , 2018 .

[13]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Ayoub Al-Hamadi,et al.  Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences , 2008, J. WSCG.

[15]  Yasushi Makihara,et al.  Gait recognition using periodic temporal super resolution for low frame-rate videos , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[16]  Yaron Caspi,et al.  Under the supervision of , 2003 .

[17]  Xiaodong Yang,et al.  Super Normal Vector for Activity Recognition Using Depth Sequences , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Peter Morguet,et al.  Spotting dynamic hand gestures in video image sequences using hidden Markov models , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[19]  Marcus A. Magnor,et al.  View and Time Interpolation in Image Space , 2008, Comput. Graph. Forum.

[20]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[21]  Zicheng Liu,et al.  HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[23]  R. Oka,et al.  Spotting Recognition of Human Gestures from Motion Images , 1994 .

[24]  Yasushi Makihara,et al.  Periodic Temporal Super Resolution Based on Phase Registration and Manifold Reconstruction , 2011, IPSJ Trans. Comput. Vis. Appl..

[25]  Arti Khaparde,et al.  Gesture recognition using DTW & piecewise DTW , 2014, 2014 International Conference on Electronics and Communication Systems (ICECS).

[26]  Ayoub Al-Hamadi,et al.  A Novel System for Automatic Hand Gesture Spotting and Recognition in Stereo Color Image , 2009, J. WSCG.

[27]  Ying Wu,et al.  Robust 3D Action Recognition with Random Occupancy Patterns , 2012, ECCV.

[28]  Stan Sclaroff,et al.  Sign Language Spotting with a Threshold Model Based on Conditional Random Fields , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  Masahiko Yachida,et al.  Video Synthesis with High Spatio-Temporal Resolution Using Motion Compensation and Spectral Fusion , 2006, IEICE Trans. Inf. Syst..