A framework for the multimodal joint work of turn construction in face-to-face interaction

Gesture increases the face-to-face communication efficacy.Face-to-face efficacy is due to parallel feedback processing from all participants.The framework explains micro-level face-to-face interaction through dataflow.The framework can be used to understand face-to-face communication loop. We propose a framework for understanding the multimodal joint work of turn construction in face-to-face interaction. Using concepts from conversation analysis, nonverbal communication, and gesture studies, in a qualitative analysis of face-to-face interaction, we observe that, collaboratively and in a joint work, participants produce moves, within the current-speaker's turn, that allow them to deal with possible moves that could compromise the projectable trajectory of the interaction in progress. Working at the micro level of interaction, we propose a framework that will allows a better understanding of how a turn can be collaboratively produced and how other levels of sequence organization can be produced in order to achieve the desired social-agreement outcome.

[1]  E. Goffman,et al.  Forms of talk , 1982 .

[2]  D. O’connell,et al.  Turn-taking: A critical analysis of the research tradition , 1990 .

[3]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .

[4]  J. Bavelas,et al.  Nonverbal and Verbal Communication: Hand Gestures and Facial Displays as Part of Language Use in Face-to-face Dialogue. , 2006 .

[5]  E. Hall,et al.  The Hidden Dimension , 1970 .

[6]  E. Schegloff Sequencing in Conversational Openings , 1968 .

[7]  Nick Llewellyn Garfinkel and Ethnomethodology , 2014 .

[8]  G. Jefferson Glossary of transcript symbols with an introduction , 2004 .

[9]  P. Niedenthal,et al.  Psychology of emotion: Interpersonal, experiential, and cognitive approaches. , 2006 .

[10]  Judith Holler,et al.  An experimental investigation of how addressee feedback affects co-speech gestures accompanying spea , 2011 .

[11]  E. Schegloff,et al.  The preference for self-correction in the organization of repair in conversation , 1977 .

[12]  Emanuel A. Schegloff,et al.  The Relevance of Repair to Syntax-for-Conversation in Discourse and Syntax. , 1979 .

[13]  Maja Pantic,et al.  Spotting agreement and disagreement: A survey of nonverbal audiovisual cues and tools , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[14]  E. Schegloff Sequence Organization in Interaction: Contents , 2007 .

[15]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[16]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[17]  Anita Pomerantz Structures of Social Action: Pursuing a response , 1985 .

[18]  Emanuel A. Schegloff,et al.  Practices and actions: Boundary cases of other‐initiated repair , 1997 .

[19]  Janet Beavin Bavelas,et al.  Using Face-to-face Dialogue as a Standard for Other Communication Systems , 1997 .

[20]  Lorenza Mondada,et al.  Understanding as an embodied, situated and sequential achievement in interaction , 2011 .

[21]  Anssi Peräkylä,et al.  Facial expression in an assessment , 2006 .

[22]  Lorenza Mondada,et al.  The local constitution of multimodal resources for social interaction , 2014 .

[23]  Charles Goodwin,et al.  Time in Action , 2002, Current Anthropology.

[24]  Jaan Valsiner Fundamentos da Psicologia Cultural: Mundos da Mente, Mundos da Vida , 2012 .

[25]  付伶俐 打磨Using Language,倡导新理念 , 2014 .

[26]  Anssi Peräkylä,et al.  Facial expression and interactional regulation of emotion , 2012 .

[27]  Jürgen Streeck,et al.  Projection and Anticipation: The Forward-Looking Nature of Embodied Communication , 2009 .

[28]  M. Hayashi,et al.  Conversational Repair and Human Understanding: Contents , 2013 .

[29]  C. Goodwin The co-operative, transformative organization of human action and knowledge , 2013 .

[30]  Adam Kendon,et al.  Semiotic diversity in utterance production and the concept of ‘language’ , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[31]  E. Schegloff,et al.  Discourse, Pragmatics, Conversation, Analysis , 1999 .

[32]  Anssi Peräkylä,et al.  Foreshadowing a problem: Turn-opening frowns in conversation , 2014 .

[33]  J. Bavelas,et al.  Listener Responses as a Collaborative Process: The Role of Gaze , 2002 .

[34]  J. Bavelas,et al.  Visible Acts of Meaning , 2000 .

[35]  M. Knapp,et al.  Nonverbal communication in human interaction , 1972 .

[36]  Mathias Broth,et al.  Walking away: The embodied achievement of activity closings in mobile interaction , 2013 .

[37]  P. Ekman Facial expression and emotion. , 1993, The American psychologist.

[38]  Milo Mileff,et al.  Head movements in the context of speech in Arabic, Bulgarian, Korean, and African-American Vernacular English , 2007 .

[39]  Craig A. Smith,et al.  Dimensions of appraisal and physiological response in emotion. , 1989, Journal of personality and social psychology.

[40]  H. Garfinkel Studies in Ethnomethodology , 1968 .

[41]  A. Kendon Some Relationships Between Body Motion and Speech , 1972 .

[42]  Harvey Sacks,et al.  Lectures on Conversation , 1995 .

[43]  Geneviève Calbris,et al.  Elements of Meaning in Gesture , 2011 .

[44]  Philippe Blache,et al.  Backchannels revisited from a multimodal perspective , 2007, AVSP.

[45]  A. Kendon Gesture: Visible Action as Utterance , 2004 .

[46]  Jakob Steensig,et al.  The Morality of Knowledge in Conversation: Acknowledgements , 2011 .

[47]  Nicole Chovil Social determinants of facial displays , 1991 .

[48]  J. Spickard Garfinkel and Ethnomethodology by John Heritage. Cambridge: Polity Press (Basil Blackwell), 1984, viii + 336 pp. $34.95 and Studies in Ethnomethodology , by Harold Garfinkel. Cambridge: Polity Press (Basil Blackwell), 1967, xi + 288 pp. $9.95 (paper, reissue) , 1987 .

[49]  Corinna Elsenbroich Relationship Thinking: Agency, Enchrony, and Human Sociality , 2015 .

[50]  L. Fleischer Telling Lies Clues To Deceit In The Marketplace Politics And Marriage , 2016 .

[51]  Harvey Sacks Explorations in the Ethnography of Speaking: An Analysis of the Course of a Joke's Telling in Conversation , 1989 .

[52]  N. J. Enfield,et al.  Language presupposes an enchronic infrastructure for social interaction , 2014 .

[53]  J. Scott Jordan Wild Systems Theory , 2023 .

[54]  Nicole Chovil Discourse‐oriented facial displays in conversation , 1991 .

[55]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 2015 .

[56]  Janet Beavin Bavelas,et al.  The psychology of facial expression: Faces in dialogue , 1997 .

[57]  Jürgen Streeck,et al.  Interaction and the living body , 2013 .

[58]  Adam Kendon,et al.  Some uses of the head shake , 2002 .

[59]  Evelyn Z. McClave Linguistic functions of head movements in the context of speech , 2000 .

[60]  D. McNeill Gesture and Thought , 2005 .

[61]  C. Creider Hand and Mind: What Gestures Reveal about Thought , 1994 .