An approach to natural gesture in virtual environments

This article presents research—an experiment and the resulting prototype—on a method for treating gestural input so that it can be used for multimodal applications, such as interacting with virtual environments. This method involves the capture and use of natural , empty-hand gestures that are made during conventional descriptive utterances. Users are allowed to gesture in a normal continuous manner, rather than being restricted to a small set of discrete gestural commands as in most other systems. The gestures are captured and analyzed into a higher-level description. This description can be used by an application-specific interpreter to understand the gestural input in its proper context. Having a gesture analyzer of this sort enables natural gesture input to any appropriate application.

[1]  Kristinn R. Thórisson,et al.  Integrating Simultaneous Input from Speech, Gaze, and Hand Gestures , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[2]  Richard A. Bolt,et al.  Multi-modal natural dialogue , 1992, CHI '92.

[3]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Karen A. Frenkel.  A conversation with Brenda Laurel , 1994, INTR.

[5]  Alan Dix.  The human interface , 1994 .

[6]  S. Kicha Ganapathy,et al.  A synthetic visual environment with hand gesturing and voice input , 1989, CHI '89.

[7]  Myron W. Krueger,et al.  Artificial reality II , 1991 .

[8]  Kaisa Väänänen,et al.  Gesture Driven Interaction as a Human Factor in Virtual Environments - An Approach with Neural Networks , 1993, Virtual Reality Systems.

[9]  Daniel T. Ling,et al.  Dialogue structures for virtual worlds , 1991, CHI.

[10]  A. Leroi‐Gourhan,et al.  Gesture and Speech , 1993 .

[11]  Edwin Bos.  Some virtues and limitations of action inferring interfaces , 1992, UIST '92.

[12]  Alan Wexelblat,et al.  A feature-based approach to continuous-gesture analysis , 1994 .

[13]  Alan Wexelblat.  Natural gesture in virtual environments , 1994 .

[14]  David Weimer,et al.  Interaction techniques using hand tracking and speech recognition , 1992 .

[15]  Richard A. Bolt,et al.  Two-handed gesture in multi-modal natural dialog , 1992, UIST '92.

[16]  Rudolf Arnheim,et al.  Hand and Mind: What Gestures Reveal About Thought by David McNeill (review) , 2017 .

[17]  Francis K. H. Quek,et al.  Toward a vision-based hand gesture interface , 1994 .

[18]  Greg Turk,et al.  Interactive simulation in a multi-person virtual world , 1992, CHI.

[19]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[20]  Norman I. Badler,et al.  Simulating humans: computer graphics animation and control , 1993 .

[21]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.