Discourse Reference Is Bimodal: How Information Status in Speech Interacts with Presence and Viewpoint of Gestures

ABSTRACT Speakers use speech and gestures to represent referents in discourse. Depending on referents’ information status, in speech speakers will vary richness of expression (e.g., lexical noun phrase [NP]/pronoun), nominal definiteness (indefinite/definite), and grammatical role (subject/object). This study tested whether these three linguistic markers of information status interact with presence of gestures and gestural viewpoint (observer/character). The results show that gestures are more frequent with less accessible referents expressed with richer spoken forms but that richness of expression does not interact with viewpoint. In contrast, nominal definiteness and grammatical role interact with both presence and viewpoint of gestures. Gestures occur mainly with indefinite lexical NPs and objects. Character viewpoint gestures occur mainly with indefinite lexical NPs and objects plus predicates. The results shed light on when and how speakers use gestures in connected discourse and specifically highlight the discursive function of gestural viewpoint.

[1]  T. Pechmann,et al.  Social interaction and the development of definite descriptions , 1982, Cognition.

[2]  T. Givon.  Functionalism and Grammar , 1995 .

[3]  Mira Ariel.  Referring and accessibility , 1988, Journal of Linguistics.

[4]  Mira Ariel,et al.  The Possessive NP Construction: Discourse Function and Discourse Profile , 2002 .

[5]  K. Yoshioka,et al.  Gesture and information structure in first and second language , 2008 .

[6]  Shanley Allen,et al.  Preferred Argument Structure in early Inuktitut spontaneous speech data , 2003 .

[7]  David McNeill,et al.  Speech, Gesture, and Discourse. , 1992 .

[8]  Ellen F. Prince,et al.  Toward a taxonomy of given-new information , 1981 .

[9]  Marianne Gullberg.  Gestures, referents, and anaphoric linkage in learner varieties , 2003 .

[10]  Peter Wittenburg,et al.  Annotation by Category: ELAN and ISO DCR , 2008, LREC.

[11]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[12]  Fey Parrill.  Viewpoint in Language: Interactions between discourse status and viewpoint in co-speech gesture , 2012 .

[13]  Asli Özyürek,et al.  Gestural Viewpoint Signals Referent Accessibility , 2013 .

[14]  M. Gullberg.  Gesture as a communication strategy in second language discourse , 1998 .

[15]  Elise Kärkkäinen,et al.  Preferred argument structure and subject role in American English conversational discourse , 1996 .

[16]  Jeanette K. Gundel,et al.  Cognitive Status and the Form of Referring Expressions in Discourse , 1993 .

[17]  D. McNeill,et al.  Abstract deixis , 1993 .

[18]  T. Givon.  Topic Continuity in Discourse , 1983 .

[19]  C. Fowler,et al.  Language and Gesture: The role of gestures and other graded language forms in the grounding of reference in perception , 2000 .

[20]  D. McNeill,et al.  Speech-gesture mismatches: Evidence for one underlying representation of linguistic and nonlinguistic information , 1998 .

[21]  William D. Marslen-Wilson,et al.  Producing Interpretable Discourse: The Establishment and Maintenance of Reference , 1982 .

[22]  W. So,et al.  Sensitivity to information status in discourse: Gesture precedes speech in unbalanced bilinguals , 2012, Applied Psycholinguistics.

[23]  Gale Stam,et al.  Thinking for speaking about motion: L1 and L2 speech and gesture , 2006 .

[24]  Marianne Gullberg,et al.  Handling Discourse: Gestures, Reference Tracking, and Communication Strategies in Early L2 , 2006 .

[25]  T. Givón,et al.  Topic continuity in discourse : a quantitative cross-language study , 1983 .

[26]  Nicholas O. Jungheim.  GESTURE AS A COMMUNICATION STRATEGY IN SECOND LANGUAGE DISCOURSE: A STUDY OF LEARNERS OF FRENCH AND SWEDISH.Marianne Gullberg. Lund, Sweden: Lund University Press, 1998. Pp. 253. , 2000, Studies in Second Language Acquisition.

[27]  Asli Özyürek,et al.  Visible Cohesion: A Comparison of Reference Tracking in Sign, Speech, and Co-Speech Gesture , 2015, Top. Cogn. Sci..

[28]  Sotaro Kita,et al.  How representational gestures help speaking , 2000 .

[29]  Matthew Stone,et al.  Modeling the Interaction between Speech and Gesture. , 1994 .

[30]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[31]  D. McNeill.  Gesture and Thought , 2005 .

[32]  Karl G. D. Bailey,et al.  Do speakers and listeners observe the Gricean Maxim of Quantity , 2006 .

[33]  Judith Holler,et al.  Chapter 22. Speakers’ use of ‘action’ and ‘entity’ gestures with definite and indefinite references , 2011 .

[34]  Janet Beavin Bavelas,et al.  Linguistic influences on gesture’s form , 2005 .

[35]  A. McCabe.  Relating events in narrative: a crosslinguistic developmental study , 1996, Journal of Child Language.

[36]  T. Chartrand,et al.  The chameleon effect: the perception-behavior link and social interaction. , 1999, Journal of personality and social psychology.

[37]  M. Argyle,et al.  A Cross-Cultural Study of the Communication of Extra-Verbal Meaning by Gesture , 1975 .

[38]  ADAM KENDON.  Some reasons for studying gesture , 1986 .

[39]  Fey Parrill.  Dual viewpoint gestures , 2009 .

[40]  Cornelia Müller,et al.  128. Gestural modes of representation as techniques of depiction , 2014 .

[41]  M. Hickmann,et al.  Cohesion and anaphora in children's narratives: a comparison of English, French, German, and Mandarin Chinese , 1999, Journal of Child Language.

[42]  Maya Hickmann,et al.  Learning to talk and gesture about motion in French , 2008 .

[43]  S. Garrod,et al.  Saying what you mean in dialogue: A study in conceptual and semantic co-ordination , 1987, Cognition.

[44]  Robert Dixon,et al.  Basic Linguistic Theory: grammatical topics , 2010 .

[45]  P. Perniss,et al.  Representations of action, motion and location in sign space: A comparison of German (DGS) and Turkish (TID) Sign Language narratives , 2008 .

[46]  M. Gullberg,et al.  Gesture production and speech fluency in competent speakers and language learners , 2013 .

[47]  GEOFFREY BEATTIE,et al.  Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation , 1999 .

[48]  John W. Du Bois.  The Discourse Basis of Ergativity , 1987 .

[49]  Mario von Cranach,et al.  Social communication and movement : studies of interaction and expression in man and chimpanzee , 1976 .

[50]  Susan Goldin-Meadow,et al.  Using the Hands to Identify Who Does What to Whom: Gesture and Speech Go Hand-in-Hand , 2009, Cogn. Sci..

[51]  Mandana Seyfeddinipur,et al.  Disfluency: Interrupting speech and gesture , 2006 .

[52]  Patricia M. Clancy,et al.  The lexicon in interaction: Developmental origins of Preferred Argument Structure in Korean , 2003 .

[53]  A. Kendon.  Some Relationships Between Body Motion and Speech , 1972 .

[54]  H. H. Clark,et al.  In search of referents for nouns and pronouns , 1979 .

[55]  W. Chafe.  Discourse, Consciousness, and Time: The Flow and Displacement of Conscious Experience in Speaking and Writing , 1996 .