Coverbal iconic gestures for object descriptions in virtual environments

This paper describes an empirical study aimed at investigating object references in Virtual Environments using iconic gestures. Observations are focused on spatial concepts conveyed gestually and their relation to features of the gesture shape. A set of important features and spatial concepts useful for automated gesture recognition is identified. Based on these findings we propose a model of an iconic reference recognizer.

[1]  R. Krauss,et al.  The Role of Speech-Related Arm/Hand Gestures in Word Retrieval , 2001 .

[2]  A. Kendon.  Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .

[3]  Yong Cao,et al.  Interactive Graphics Design with Situated Agents , 1993, Graphics and Robotics.

[4]  Sotaro Kita,et al.  Movement Phase in Signs and Co-Speech Gestures, and Their Transcriptions by Human Coders , 1997, Gesture Workshop.

[5]  Kristinn R. Thórisson,et al.  Integrating Simultaneous Input from Speech, Gaze, and Hand Gestures , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[6]  Pierre Feyereisen,et al.  The Meaning of Gestures - What Can Be Understood Without Speech , 1988 .

[7]  Uri Hadar,et al.  Two Types of Gesture and Their Role in Speech Production , 1989 .

[8]  A. Leroi‐Gourhan,et al.  Gesture and Speech , 1993 .

[9]  B. Butterworth,et al.  Iconic gestures, imagery, and word retrieval in speech , 1997 .

[10]  A. Kendon.  Do Gestures Communicate? A Review , 1994 .

[12]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[13]  Philip R. Cohen,et al.  QuickSet: multimodal interaction for distributed applications , 1997, MULTIMEDIA '97.

[14]  R. Battison,et al.  Lexical Borrowing in American Sign Language , 1978 .

[15]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[16]  Ipke Wachsmuth,et al.  Gesture and Sign Language in Human-Computer Interaction , 1997, Lecture Notes in Computer Science.

[17]  C. Hartshorne,et al.  Collected Papers of Charles Sanders Peirce , 1935, Nature.

[18]  GEOFFREY BEATTIE,et al.  Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation , 1999 .

[19]  Marc Erich Latoschik,et al.  Utilize speech and gestures to realize natural interaction in a virtual environment , 1998, IECON '98. Proceedings of the 24th Annual Conference of the IEEE Industrial Electronics Society (Cat. No.98CH36200).