Inertial Motion Sensing Glove for Sign Language Gesture Acquisition and Recognition

The most popular systems for automatic sign language recognition are based on vision. They are user-friendly, but very sensitive to changes in regard to recording conditions. This paper presents a description of the construction of a more robust system-an accelerometer glove-as well as its application in the recognition of sign language gestures. The basic data regarding inertial motion sensors and the design of the gesture acquisition system as well as project proposals are presented. The evaluation of the solution presents the results of the gesture recognition attempt by using a selected set of sign language gestures with a described method based on Hidden Markov Model (HMM) and parallel HMM approaches. The proposed usage of parallel HMM for sensor-fusion modeling reduced the equal error rate by more than 60%, while preserving 99.75% recognition accuracy.

[1]  Hervé Bourlard,et al.  Subband-based speech recognition , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  Hervé Bourlard,et al.  A mew ASR approach based on independent processing and recombination of partial frequency bands , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[3]  Rini Akmeliawati,et al.  Hidden Markov model for human to computer interaction: a study on human hand gesture recognition , 2011, Artificial Intelligence Review.

[4]  Dimitris N. Metaxas,et al.  A Framework for Recognizing the Simultaneous Aspects of American Sign Language , 2001, Comput. Vis. Image Underst..

[5]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[6]  H. Teleb,et al.  Data glove integration with 3D virtual environments , 2012, 2012 International Conference on Systems and Informatics (ICSAI2012).

[7]  Petros Maragos,et al.  Dynamic-static unsupervised sequentiality, statistical subunits and lexicon for sign language recognition , 2014, Image Vis. Comput..

[8]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[9]  Yao-Chiang Kan,et al.  A Wearable Inertial Sensor Node for Body Motion Analysis , 2012, IEEE Sensors Journal.

[10]  P. M. Mendes,et al.  FBG Sensing Glove for Monitoring Hand Posture , 2011, IEEE Sensors Journal.

[11]  Guang-Zhong Yang,et al.  Development of a Wireless Sensor Glove for Surgical Skills Assessment , 2009, IEEE Transactions on Information Technology in Biomedicine.

[12]  W. Stokoe,et al.  Sign language structure: an outline of the visual communication systems of the American deaf. 1960. , 1961, Journal of deaf studies and deaf education.

[13]  Mohammed Waleed Kadous,et al.  Temporal classification: extending the classification paradigm to multivariate time series , 2002 .

[14]  Surendra Ranganath,et al.  Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  E. Dijkstra.  UPPER LIMB PROJECT Modeling of the upper limb , 2022 .

[16]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[17]  Nasser Kehtarnavaz,et al.  A Real-Time Human Action Recognition System Using Depth and Inertial Sensor Fusion , 2016, IEEE Sensors Journal.

[18]  Yunhui Liu,et al.  2D Human Gesture Tracking and Recognition by the Fusion of MEMS Inertial and Vision Sensors , 2014, IEEE Sensors Journal.

[19]  Frederico G. Guimarães,et al.  Feature extraction in Brazilian Sign Language Recognition based on phonological structure and using RGB-D sensors , 2014, Expert Syst. Appl..

[20]  Steve Young,et al.  The HTK book version 3.4 , 2006 .

[21]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[22]  Nasser Kehtarnavaz,et al.  Fusion of Inertial and Depth Sensor Data for Robust Hand Gesture Recognition , 2014, IEEE Sensors Journal.

[23]  Asha,et al.  A Hand Gesture Recognition Framework and Wearable Gesture Based Interaction Prototype for Mobile Devices , 2015 .

[24]  Karl-Friedrich Kraiss,et al.  Recent developments in visual sign language recognition , 2008, Universal Access in the Information Society.

[25]  Huosheng Hu,et al.  Applications of wearable inertial sensors in estimation of upper limb movements , 2006, Biomed. Signal Process. Control..