Recognition-based gesture spotting in video games

In vision-based interfaces for video games, gestures are used as commands for the games instead of pressing buttons on a keyboard or moving a mouse. In these interfaces, unintentional movements and continuous gestures must be supported to provide the user with a more natural interface. This paper proposes a novel gesture spotting method that combines gesture spotting with gesture recognition. It recognizes the meaningful movements while concurrently separating unintentional movements from a given image sequence. We applied our method to the recognition of upper-body gestures as the interface between a video game (Quake II) and its user. Experimental results show that the proposed method is, on average, 93.36% reliable in spotting gestures from continuous gestures, confirming its potential for a gesture-based interface for computer games.

[1]  A. Lecours,et al.  The Biological foundations of gestures : motor and semiotic aspects , 1986 .

[2]  Hang Joon Kim,et al.  Segmentation of touching characters using an MLP , 1998, Pattern Recognit. Lett..

[3]  Vladimir Pavlovic,et al.  Speech/Gesture Interface to a Visual-Computing Environment , 2000, IEEE Computer Graphics and Applications.

[4]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[5]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Hang Joon Kim,et al.  A Clustering Approach to the Vision-Based Interface for Interactive Computer Games , 2003, IDEAL.

[7]  R. Oka,et al.  Towards the integration of spontaneous speech and gesture based on spotting method , 1996, 1996 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems (Cat. No.96TH8242).

[8]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[9]  Dariu Mihai Gavrila,et al.  Vision-based 3-D tracking of humans in action , 1998 .

[10]  Ho-Sub Yoon,et al.  Gesture-based editing system for graphic primitives and alphanumeric characters , 1999 .

[11]  Hang Joon Kim,et al.  Design of an interface on PDA for Korean , 2000, IEEE Trans. Consumer Electron..

[12]  Mubarak Shah,et al.  Visual gesture recognition , 1994 .

[13]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Francis K. H. Quek,et al.  Toward a vision-based hand gesture interface , 1994 .

[15]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Michael J. Black,et al.  Recognizing temporal trajectories using the condensation algorithm , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[17]  Hang Joon Kim,et al.  Real-Time Face Detection and Tracking Using PCA and NN , 2002, PRICAI.

[18]  Hang Joon Kim,et al.  On-line recognition of cursive Korean characters using graph representation , 2000, Pattern Recognit..

[19]  Paul A. Beardsley,et al.  Computer Vision for Interactive Computer Graphics , 1998, IEEE Computer Graphics and Applications.