Studies in automated hand gesture analysis: an overview of functional types and gesture phases

Abstract This paper presents an overview of studies on automated hand gesture analysis, which is mainly concerned with recognition and segmentation issues related to functional types and gesture phases. The issues selected for discussion have been arranged in a way that takes account of problems within the Theory of Gestures that each study seeks to address. Their principal computational factors that were involved in conducting the analysis of automated hand gesture have been examined, and an analysis of open research issues has been carried out for each application dealt with in the studies.

[1]  Rashid Ansari,et al.  Multimodal human discourse: gesture and speech , 2002, TCHI.

[2]  Mary P. Harper,et al.  Multimodal model integration for sentence unit detection , 2004, ICMI '04.

[3]  Francis K. H. Quek,et al.  Catchments, prosody and discourse , 2001 .

[4]  J. Hampton.  An investigation of the nature of abstract concepts , 1981, Memory & cognition.

[5]  Mohammed Yeasin,et al.  Prosody based audiovisual coanalysis for coverbal gesture recognition , 2005, IEEE Transactions on Multimedia.

[6]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[7]  Helton Hideraldo Bíscaro,et al.  Hand movement recognition for Brazilian Sign Language: A study using distance-based neural networks , 2009, 2009 International Joint Conference on Neural Networks.

[8]  E. Krahmer,et al.  Gesturing by speakers with aphasia: how does it compare? , 2013, Journal of speech, language, and hearing research : JSLHR.

[9]  Julie Carson-Berndsen.  Time Map Phonology , 1998 .

[10]  Huosheng Hu,et al.  Action classification of 3D human models using dynamic ANNs for mobile robot surveillance , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[11]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  G. Baranek,et al.  Analysis of Social Interaction Gestures in Infants with Autism , 2006, Child neuropsychology : a journal on normal and abnormal development in childhood and adolescence.

[13]  Scott K. Liddell.  Grammar, Gesture, and Meaning in American Sign Language , 2003 .

[14]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[15]  Costanza Navarretta,et al.  The MUMIN multimodal coding scheme , 2005 .

[16]  Manolya Kavakli,et al.  A survey of speech-hand gesture recognition for the development of multimodal interfaces in computer games , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[17]  Harry Bunt,et al.  Dimensions of communication , 2009 .

[18]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[19]  Rachel E. Scherr.  Gesture Analysis for Physics Education Researchers. , 2007, 0712.0581.

[20]  Justine Cassell,et al.  Recovering the temporal structure of natural gesture , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[21]  Alan Garnham,et al.  Psycholinguistics: Central Topics , 1985 .

[22]  Timo Sowa,et al.  The Recognition and Comprehension of Hand Gestures - A Review and Research Agenda , 2006, ZiF Workshop.

[23]  Julie Carson-Berndsen,et al.  Finite State Models and Event Logics in Speech Recognition , 1997 .

[24]  Francis K. H. Quek,et al.  Hand gesture symmetric behavior detection and analysis in natural conversation , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[25]  Francis Quek,et al.  Gesture cues for conversational interaction in monocular video , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[26]  Sarajane Marques Peres,et al.  Gesture unit segmentation using support vector machines: segmenting gestures from rest positions , 2013, SAC '13.

[27]  Dafydd Gibbon,et al.  Gesture Theory is Linguistics: On Modelling Multimodality as Prosody , 2009, PACLIC.

[28]  Mary P. Harper,et al.  Gesture patterns during speech repairs , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[29]  Sanshzar Kettebekov.  Exploiting prosodic structuring of coverbal gesticulation , 2004, ICMI '04.

[30]  Sethuraman Panchanathan,et al.  Automated gesture segmentation from dance sequences , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[31]  Ron Artstein,et al.  Survey Article: Inter-Coder Agreement for Computational Linguistics , 2008, CL.

[32]  Fuxing Yang,et al.  The Talking Hands?: The Relation between Gesture and Language in Aphasic Patients , 2011 .

[33]  Francis K. H. Quek.  The Catchment Feature Model: A Device for Multimodal Fusion and a Bridge between Signal and Sense , 2004, EURASIP J. Adv. Signal Process..

[34]  Peter Bull,et al.  The Use of Hand Gesture in Political Speeches: Some Case Studies , 1986 .

[35]  Regina Barzilay,et al.  Gesture Salience as a Hidden Variable for Coreference Resolution and Keyframe Extraction , 2008, J. Artif. Intell. Res..

[36]  Emiel Krahmer,et al.  Gesturing by aphasic speakers, how does it compare? , 2011, CogSci.

[37]  Mitchell P. Marcus,et al.  Form: an experiment in the annotation of the kinetics of gesture , 2005 .

[38]  C. A. Pickering.  The search for a safer driver interface: a review of gesture recognition human machine interface , 2005 .

[39]  Mohammed Waleed Kadous,et al.  Temporal classification: extending the classification paradigm to multivariate time series , 2002 .

[40]  Dafydd Gibbon,et al.  A computational model of arm gestures in conversation , 2003, INTERSPEECH.

[41]  Mark T. Maybury,et al.  Multimedia Annotation, Querying, and Analysis in Anvil , 2011 .

[42]  Craig H. Martell,et al.  Corpus-Based Gesture Analysis: an Extension of the Form Dataset for the Automatic Detection of Phases in a Gesture , 2007, Int. J. Semantic Comput..

[43]  Gilles Fauconnier,et al.  Conceptual Blending, Form and Meaning , 2003 .

[44]  Francis K. H. Quek,et al.  Hand Motion Gesture Frequency Properties and Multimodal Discourse Analysis , 2006, International Journal of Computer Vision.

[45]  A. Kendon.  Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .

[46]  Dafydd Gibbon.  Time Types and Time Trees: Prosodic Mining and Alignment of Temporally Annotated Data , 2006 .

[47]  Sotaro Kita,et al.  Movement Phase in Signs and Co-Speech Gestures, and Their Transcriptions by Human Coders , 1997, Gesture Workshop.

[48]  Michael Neff,et al.  Segmentation of hand gestures using motion capture data , 2013, AAMAS.

[49]  Craig Martell.  FORM: An Extensible, Kinematically-based Gesture Annotation Scheme , 2002, LREC.

[50]  J. Lyons.  Semantics: Index of personal names , 1977 .

[51]  Nicla Rossini.  The Analysis of Gesture: Establishing a Set of Parameters , 2003, Gesture Workshop.

[52]  Hennie Brugman,et al.  Annotating Multi-media/Multi-modal Resources with ELAN , 2004, LREC.

[53]  Eric Margolis,et al.  A reassessment of the shift from the classical theory of concepts to prototype theory , 1994, Cognition.

[54]  Marino Bonaiuto,et al.  Coding Hand Gestures: A Reliable Taxonomy and a Multi-media Support , 2011, COST 2102 Training School.

[55]  A. B. M. Shawkat Ali,et al.  HMM based hand gesture recognition: A review on techniques and approaches , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[56]  D. McNeill.  Gesture and Thought , 2005 .

[57]  Przemyslaw Lenkiewicz,et al.  Towards Automatic Gesture Stroke Detection , 2012, LREC.