3 D versus 2 D Pose Information for Recognition of NGT Signs

In this article, we evaluate the improvement in sign recognition that can be achieved with 3D tracking, compared to recognition with image plane tracking. Experiments are shown using a pair of stereo cameras, from which 3D positions of the segmented hands are estimated by triangulation between the left and right cameras. A sign classifier is trained on a set of 37 different NGT signs using 2D and 3D features. The results show that 3D features improve sign detection results, especially when 3D features are only used when their relevance is known beforehand. Furthermore, we show the positive effect of perspective distortion at close range.

[1]  David J. Kriegman,et al.  A Real-Time Approach to the Spotting, Representation, and Recognition of Hand Gestures for Human-Computer Interaction , 2002, Comput. Vis. Image Underst..

[2]  Thad Starner,et al.  A gesture-based american sign language game for deaf children , 2005, CHI Extended Abstracts.

[3]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Dimitris N. Metaxas,et al.  ASL recognition based on a coupling between HMMs and 3D motion analysis , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[5]  J. Ohya,et al.  Automatic skin-color distribution extraction for face detection and tracking , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[6]  Shaogang Gong,et al.  Tracking and segmenting people in varying lighting conditions using colour , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[7]  Manolis I. A. Lourakis,et al.  Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera , 2004, ECCV.

[8]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[9]  David Windridge,et al.  A Linguistic Feature Vector for the Visual Interpretation of Sign Language , 2004, ECCV.

[10]  Sebastian Lang,et al.  Improving adaptive skin color segmentation by incorporating results from face detection , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.