An analysis of modal (pragmatic) functions of gesture

When humans communicate naturally, a lot more is transferred than just the semantic content. The meaning of an utterance is enhanced by verbal pragmatic markers but also by gestural and other non-verbal signals in order to classify the semantic content of the utterances (Wharton, 2009). Senders want to communicate their convictions, viewpoints, knowledge, attitudes, among others. These signals are not discourse related, they merely support the recipient to arrive at the correct interpretation that was intended by the sender. Recipients perceive those signals overlaid to the semantic content and integrate everything into a congruent message. We define such signals asmodal (pragmatic) functions1, a sub-category of pragmatic functions. This notion is related to Kendon’s (2004)modal functions which “seem to operate on a given unit of verbal discourse and show how it is to be interpreted” (p. 225) as, e.g., to “indicate what units are ‘focal’ for their arguments” (Kendon, 1995, p. 276). Aside from individual articles, a recent summary of gestures that take up pragmatic functions by Payrató and Teßendorf (2013) mentions various gestures with certain recurrent form features/gesture families and the corresponding pragmatic functions, e.g., away gestures (Bressem & Müller, 2014, p. 1599) “to mark arguments, ideas, and actions as uninteresting and void”. Our focus is particularly on how gestures realise the functions and, thus, highlight, downplay or quantify a unit of an utterance. We advocate a full account of modal (pragmatic) functions tackled in an empirical approach and with the necessary amount of detail, partly carried out in our previous work (Freigang & Kopp, 2015, 2016). The scope of the modal markers can be identified by co-occurring prosodical cues, since there are also modal and a↵ective tones in prosody (Lu, Aubergé, & Rilliard, 2012) and nuclear accents align with gesture strokes (Ebert, Evert, & Wilmes, 2011). However, “no clear notion of pragmatic gesture is available, neither in the area of (linguistic) pragmatics nor in gesture studies” (Payrató & Teßendorf, 2013, p. 1536) and discussing the mappings between modal (pragmatic) functions in gesture and existing linguistic frameworks is even a step further. In gesture research, terms such as speech acts (illocutionary markers in particular), communicative intend and interpersonal markers are employed, however, they cannot be related to gestures as strongly as on a lexical, grammatical or speech level. Mello and Raso (2011) studied the linguistic categories illocution, modality (epistemic, deontic, etc., for more details cf. e.g. Petukhova and Bunt (2010)) and attitude with special attention to the pragmatics-prosody interface. In an experiment, they tested the e↵ects of changed prosodic markers and found that