Sign language recognition using model-based tracking and a 3 D Hopfield neural network

This paper presents a sign language recognition system which consists of three modules: model-based hand tracking, feature extraction, and gesture recognition using a 3D Hopfield neural network (HNN). The first one uses the Hausdorff distance measure to track shapevariant hand motion, the second one applies the scale and rotation-invariant Fourier descriptor to characterize hand figures, and the last one performs a graph matching between the input gesture model and the stored models by using a 3D modified HNN to recognize the gesture. Our system tests 15 different hand gestures. The experimental results show that our system can achieve above 91% recognition rate, and the recognition process time is about 10 s. The major contribution in this paper is that we propose a 3D modified HNN for gesture recognition which is more reliable than the conventional methods.

[1]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[2]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Takeo Kanade,et al.  DigitEyes: vision-based hand tracking for human-computer interaction , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[4]  Chung-Lin Huang,et al.  Motion‐compensated interpolation for scan rate up‐conversion , 1996 .

[5]  Mubarak Shah,et al.  Visual gesture recognition , 1994 .

[6]  Daniel P. Huttenlocher,et al.  Comparing images using the Hausdorff distance under translation , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Shinichi Tamura,et al.  Recognition of sign language motion images , 1988, Pattern Recognit..

[8]  Ralph Roskies,et al.  Fourier Descriptors for Plane Closed Curves , 1972, IEEE Transactions on Computers.

[9]  Andrew D. Wilson,et al.  Using Connguration States for the Representation and Recognition of Gesture 2 Related Work , 1995 .

[10]  A E Marble,et al.  Image processing system for interpreting motion in American Sign Language. , 1992, Journal of biomedical engineering.

[11]  King-Sun Fu,et al.  Shape Discrimination Using Fourier Descriptors , 1977, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Birgit Ueberreiter,et al.  Pattern Recognition by Graph Matching - Combinatorial versus continuous Optimization , 1988, Int. J. Pattern Recognit. Artif. Intell..

[13]  Daniel P. Huttenlocher,et al.  Tracking non-rigid objects in complex scenes , 1993, 1993 (4th) International Conference on Computer Vision.

[14]  Malayappan Shridhar,et al.  High accuracy character recognition algorithm using fourier and topological descriptors , 1984, Pattern Recognit..

[15]  Nasser M. Nasrabadi,et al.  Object recognition by a Hopfield neural network , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[16]  Vladimir Pavlovic,et al.  Hand Gesture Modeling, Analysis, and Synthesis , 1995 .

[17]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[18]  P. Anandan,et al.  Optimization in Model Matching and Perceptual Organization , 1989, Neural Computation.

[19]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[20]  Yuntao Cui,et al.  Learning-based hand sign recognition using SHOSLIF-M , 1995, Proceedings of IEEE International Conference on Computer Vision.