New insights into the medium hand: Discovering recurrent structures in gestures

Abstract This paper shows that an analysis of gestures based on the four parameters hand shape, orientation, movement, and position in gesture space, introduced for the notation of sign language (Battison 1974; Stokoe 1960) can uncover recurrent patterns and structures in gestures. By presenting two studies we argue for a discovery procedure that rests upon a systematic description of gestures applying the four parameters and upon a clear discrimination between form and meaning in the analytical process. Using this methodological framework it is possible to uncover clusters of recurrent forms and a systematic variation of form and meaning in a recurrent gesture.

[1]  Streeck.  The Significance of Gesture: How it is Established. , 1988 .

[2]  W. Stokoe.  Sign Language Structure , 1980 .

[3]  Stefan Kopp,et al.  Towards integrated microplanning of language and iconic gesture for multimodal output , 2004, ICMI '04.

[4]  Mark Rowlands,et al.  The body in mind , 1999 .

[5]  Peter Muntigl.  Modelling Multiple Semiotic Systems: The Case of Gesture and Speech , 2004 .

[6]  Vyvyan Evans,et al.  Cognitive Linguistics: An Introduction , 2001 .

[7]  W. Stokoe,et al.  A dictionary of American sign language on linguistic principles , 1965 .

[8]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[9]  GEOFFREY BEATTIE,et al.  Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation , 1999 .

[10]  Cornelia Müller,et al.  Redebegleitende Gesten : Kulturgeschichte, Theorie, Sprachvergleich , 1998 .

[11]  Paola Pietrandrea,et al.  Verbal and Signed Languages: Comparing Structures, Constructs and Methodologies , 2007 .

[12]  Ronald W. Langacker,et al.  Cognitive Grammar: A Basic Introduction , 2008 .

[13]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[14]  Ipke Wachsmuth,et al.  Interpretation of Shape-Related Iconic Gestures in Virtual Environments , 2001, Gesture Workshop.

[15]  Ernest W.B. Hess-Lüttich.  Ellen Fricke, Origo, Geste und Raum, Lokaldeixis im Deutschen, Linguistik, Impulse und Tendenzen, 24, Berlin, Walter de Gruyter, 2007 [Rezension] , 2008 .

[16]  D. Alan Cruse,et al.  Structuring of word meaning I: An overview , 2002 .

[17]  D. McNeill.  So you think gestures are nonverbal , 1985 .

[18]  Irene Mittelberg.  Methodology for multimodality: One way of working with speech and gesture data , 2007 .

[19]  Susan Duncan,et al.  Gesture, verb aspect, and the nature of iconic imagery in natural discourse , 2002 .

[20]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[21]  Fey Parrill.  Hand To Mouth: Linking Spontaneous Gesture and Aspect , 2002 .

[22]  R. Battison,et al.  Lexical Borrowing in American Sign Language , 1978 .

[23]  G. Lakoff,et al.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[24]  CAROL M. SPARHAWK.  Contrastive-Identificational Features of Persian Gesture , 1978 .

[25]  J. Prost,et al.  Gesture in Naples and Gesture in Classical Antiquity , 2001 .

[26]  Isabella Poggi.  Symbolic Gestures. The case of the Italian Gestionary , 2002 .

[27]  Hannes Rieser,et al.  On Factoring Out a Gesture Typology from the Bielefeld Speech-and-Gesture-Alignment Corpus (SAGA) , 2009, Gesture Workshop.

[28]  Roland Pfau,et al.  Grammaticalization in sign languages , 2011 .

[29]  Sherman Wilcox,et al.  Cross-linguistic and historical data from signed languages , 2004 .

[30]  Geneviève Calbris.  From left to right...: Coverbal gestures and their symbolic use of space , 2008 .

[31]  H. Brookes.  A Repertoire of South African Quotable Gestures , 2004 .

[32]  A. Kendon.  Gestures as illocutionary and discourse structure markers in Southern Italian conversation , 1995 .

[33]  Michael Kipp,et al.  Gesture generation by imitation: from human behavior to computer character animation , 2005 .

[34]  Cornelia Müller,et al.  Metaphors Dead and Alive, Sleeping and Waking: A Dynamic View , 2008 .

[35]  Isabella Poggi.  Iconicity in different types of gestures , 2008 .

[36]  Onno Crasborn,et al.  Phonetic implementation of phonological categories in Sign Language of the Netherlands , 2001 .

[37]  Rebecca A. Webb.  Linguistic features of metaphoric gestures , 1997 .

[38]  Joel Sherzer.  The Brazilian Thumbs‐Up Gesture , 1991 .

[39]  R. Langacker.  Foundations of cognitive grammar , 1983 .

[40]  Matthew Stone,et al.  A Formal Semantic Analysis of Gesture , 2009, J. Semant..

[41]  The relationship between verbal and gestural contributions in conversation: A comparison of three methods , 2009 .

[42]  A. Kendon.  An Agenda for Gesture Studies , 2007 .

[43]  Susan Duncan,et al.  Growth points in thinking-for-speaking , 1998 .

[44]  Geneviève Calbris,et al.  From cutting an object to a clear cut analysis: Gesture as the representation of a preconceptual schema linking concrete actions to abstract notions , 2003 .

[45]  Adam Kendon,et al.  Some reflections on the relationship between ‘gesture’ and ‘sign’ , 2008 .

[46]  Martha W. Alibali,et al.  I see it in my hands’ eye: Representational gestures reflect conceptual demands , 2007 .

[47]  Adam Kendon,et al.  How gestures can become like words , 1988 .

[48]  Ellen Fricke.  Origo, Geste und Raum : Lokaldeixis im Deutschen , 2007 .

[49]  Ellen Fricke.  Phonaestheme, Kinaestheme und multimodale Grammatik: Wie Artikulationen zu Typen werden, die bedeuten können , 2010 .

[50]  R. Battison,et al.  Phonological Deletion in American Sign Language , 2013 .

[51]  G. Lakoff.  Women, fire, and dangerous things : what categories reveal about the mind , 1989 .

[52]  Sedinha Teßendorf,et al.  117. Pragmatic and metaphoric – combining functional with cognitive approaches in the analysis of the “brushing aside gesture” , 2014 .

[53]  Anne Lord.  From left to right , 2005 .

[54]  Jana Bressem,et al.  Rethinking gesture phases: Articulatory features of gestural movement? , 2011 .

[55]  D. McNeill.  Hand and Mind , 1995 .

[56]  Jana Bressem,et al.  Rezension zu: Ulrike Bohle: Das Wort ergreifen - das Wort übergeben. Explorative Studie zur Rolle redebegleitender Gesten in der Organisation des Sprecherwechsels , 2009 .

[57]  Hedda Lausberg,et al.  Methods in Gesture Research: , 2009 .

[58]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[59]  Jana Bressem,et al.  70. A linguistic perspective on the notation of form features in gestures , 2013 .

[60]  N. Presmeg.  The body in the mind: The bodily basis of meaning, imagination and reason , 1992 .

[61]  Silva H. Ladewig.  Beschreiben, suchen und auffordern — Varianten einer rekurrenten Geste , 2010 .

[62]  D. McNeill.  Gesture and Thought , 2005 .

[63]  Mitchell P. Marcus,et al.  Form: an experiment in the annotation of the kinetics of gesture , 2005 .

[64]  W. Stokoe,et al.  Semiotics and Human Sign Languages , 1972 .

[65]  Eve Sweetser,et al.  What we mean by meaning: Conceptual integration in gesture analysis and transcription , 2005 .

[66]  Stefan Kopp,et al.  Verbal or Visual? How Information is Distributed across Speech and Gesture in Spatial Dialog , 2006 .

[67]  A. Scheflen.  How Behavior Means , 1972 .

[68]  Janet Beavin Bavelas,et al.  An experimental study of when and how speakers use gestures to communicate , 2002 .

[69]  Carolyn G. Fidelman,et al.  The semiotics of French gestures , 1990 .

[70]  Ellen Fricke,et al.  Grammatik multimodal : wie Wörter und Gesten zusammenwirken , 2012 .

[71]  David McNeill,et al.  Body – Language – Communication: An International Handbook on Multimodality in Human Interaction , 2013 .

[72]  R. Birdwhistell.  Kinesics and context , 1970 .

[73]  Lluís Payrató,et al.  A pragmatic view on autonomous gestures: A first repertoire of Catalan emblems , 1993 .

[74]  Jean-Pierre Koenig,et al.  Metaphoric gestures and some of their relations to verbal metaphoric expressions , 1998 .

[75]  Jana Bressem,et al.  45. Towards a grammar of gestures: A form-based view , 2013 .

[76]  Dirk Geeraerts,et al.  The structure of lexical variation: Meaning, naming, and context , 1994 .

[77]  Wendy Sandler,et al.  Sign Language and Linguistic Universals: Entering the lexicon: lexicalization, backformation, and cross-modal borrowing , 2006 .

[78]  C. Müller.  Wie Gesten bedeuten. Eine kognitiv-linguistische und sequenzanalytische Perspektive , 2010 .

[79]  E. Klima.  The signs of language , 1979 .