Corpus-Based Gesture Analysis: an Extension of the Form Dataset for the Automatic Detection of Phases in a Gesture

We present the results of using an extension of the FORM gesture dataset to predict the mid-level phenomenon of phase. We compare the results of human phase prediction with automated prediction using machine-learning techniques. Specifically, we present the results of hidden Markov model experiments using an extended version of the FORM data to predict phase labels. Additionally, we compare FORM to the currently most accurate method of data gathering in this field — motion capture — by comparing the predictive accuracy of the physical gesture models produced by FORM and by motion capture for phase labeling.

[1]  Tracy L. Westeyn,et al.  Georgia tech gesture toolkit: supporting experiments in gesture recognition , 2003, ICMI '03.

[2]  A. Kendon.  Some Relationships Between Body Motion and Speech , 1972 .

[3]  J. P. Foley,et al.  Gesture and Environment , 1942 .

[4]  D. Efron,et al.  Gesture, race and culture : a tentative study of the spatio-temporal and "linguistic" aspects of the gestural behavior of eastern Jews and southern Italians in New York City, living under similar as well as different environmental conditions , 1942 .

[5]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[6]  Francis K. H. Quek,et al.  Gestural Origo and Loci-Transitions in Natural Discourse Segmentation , 2001 .

[7]  F ChenStanley,et al.  An Empirical Study of Smoothing Techniques for Language Modeling , 1996, ACL.

[8]  Adam Kendon,et al.  How gestures can become like words , 1988 .

[9]  Peter Wittenburg,et al.  Multimodal Annotations in Gesture and Sign Language Studies , 2002, LREC.

[10]  Charles Elkan,et al.  Using the Triangle Inequality to Accelerate k-Means , 2003, ICML.

[11]  Mitchell P. Marcus,et al.  Form: an experiment in the annotation of the kinetics of gesture , 2005 .

[12]  Stanley F. Chen,et al.  An empirical study of smoothing techniques for language modeling , 1999 .

[13]  Sven C. Martin,et al.  Statistical Language Modeling Using Leaving-One-Out , 1997 .

[14]  Steve Young,et al.  The HTK hidden Markov model toolkit: design and philosophy , 1993 .