Gesture-based editing system for graphic primitives and alphanumeric characters

Abstract This paper presents a system that edits graphic primitives and alphanumeric characters using hand gestures in natural environments. Gesture is one of the most natural means of enhancing human-computer interaction (HCI) techniques to the level of human communication. This research aims to recognize one-stroke pictorial gestures from visual images, and to develop a graphic/text editing system running in real time. The tasks are performed through three steps: moving-hand tracking and trajectory generation, key-gesture segmentation, and gesture recognition by analyzing dynamic features. A gesture vocabulary consists of 48 gestures of three types: (1) six editing commands; (2) six graphic primitives; and (3) alphanumeric characters—26 alphabetic and 10 numerical. A meaningful gesture part is segmented by a spotter using the phase-based velocity constraints. Some dynamic features are obtained from spatio-temporal trajectories and quantized by the K-means algorithm. The quantized vectors were trained and tested using hidden Markov models (HMMs). Experimental results show about 88.5% correct recognition rate, and also the practical potential for applying these techniques to several areas such as the remote control of robots or of electronic household appliances, or object manipulation in VR systems.

[1]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[2]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[3]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[4]  W. Klein,et al.  Speech, place, and action : studies in deixis and related topics , 1982 .

[5]  Ryuichi Oka,et al.  A Proposal of Pattern Space Trajectory for Gesture Spotting Recognition , 1996, BMVC.

[6]  James W. Davis,et al.  Action Recognition Using Temporal Templates , 1997 .

[7]  Ho-Sub Yoon,et al.  Hand gesture recognition using hidden Markov models , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[8]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Roger B. Dannenberg,et al.  Multimedia interface design , 1992 .

[10]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[11]  Ferdinando Silvestro Samaria,et al.  Face recognition using Hidden Markov Models , 1995 .

[12]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[13]  Kirsti Grobel,et al.  Real-time hand-arm motion analysis using a single video camera , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[14]  Mubarak Shah,et al.  Motion-Based Recognition , 1997, Computational Imaging and Vision.

[15]  Francis K. H. Quek,et al.  Toward a vision-based hand gesture interface , 1994 .

[16]  A. Lecours,et al.  The Biological foundations of gestures : motor and semiotic aspects , 1986 .

[17]  Yangsheng Xu,et al.  Human action learning via hidden Markov model , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[18]  Beth Levy,et al.  Conceptual Representations in Lan-guage Activity and Gesture , 1980 .

[19]  Myron W. Krueger,et al.  Artificial reality II , 1991 .

[20]  Fumio Miyazaki,et al.  Description and recognition of human gestures based on the transition of curvature from motion images , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.