Analysis of haptic data for sign language recognition

For the past two years we have been addressing the challenges involved in managing the data generated within immersive environments. We together with many other researchers have addressed the management of obvious data types such as image, video, audio and text. However, we identified a set of less familiar data types, collectively termed as immersidata, that are specific to immersive environments. In this paper, we focus our attention on analysis of a kind of immersidata known as haptic data. We propose to analyze the haptic data acquired from CyberGlove to recognize different static hand signs automatically. The ultimate objective is to understand how to model and store haptic data in a database, for similar types of applications. We propose several techniques to analyze subtle changes in hand signs and words (a series of signs). We show that our techniques can recognize the most important features to distinguish between two letters and several preliminary experiments demonstrate more than 84.66% accuracy in sign recognition for a 10-sign vocabulary.

[1]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[2]  Charles Elkan,et al.  Boosting and Naive Bayesian learning , 1997 .

[3]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[4]  Daw-Tung Lin.  Spatio-temporal hand gesture recognition using neural networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[5]  Yangsheng Xu,et al.  Online, interactive learning of gestures for human/robot interfaces , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[6]  Geoffrey E. Hinton,et al.  Glove-TalkII: an adaptive gesture-to-formant interface , 1995, CHI '95.

[7]  Anders Sandberg,et al.  Gesture Recognition using Neural Networks , 1997 .

[8]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[9]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[10]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[11]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[12]  Masumi Ishikawa,et al.  Recognition of a hand-gesture based on self-organization using a DataGlove , 1999, ICONIP'99. ANZIIS'99 & ANNES'99 & ACNN'99. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378).

[13]  Cyrus Shahabi,et al.  Immersidata Management: Challenges in Management of Data Generated within an Immersive Environment , 1999, Multimedia Information Systems.

[14]  Pedro M. Domingos,et al.  Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier , 1996, ICML.

[15]  KwangYun Wohn,et al.  Recognition of space-time hand-gestures using hidden Markov model , 1996, VRST.

[16]  Lingling Zhang,et al.  Alternative techniques for the efficient acquisition of haptic data , 2001, SIGMETRICS '01.