Speaker-gaze Modulates the Inter-personal Repetition of Hand Gestures

One study found that observers retained more information from hand gestures that speakers gazed at, possibly because speaker-gaze shifted observers' attention covertly. Speaker-gaze may thus modulate the role of gestures in communication. One hypothesized communicative function of gestures, and specifically of the inter-personal repetition of gestures, is to facilitate the process of creating common ground (grounding). Therefore, speaker-gaze may also influence the inter-personal repetition of gestures. In an experimental study, we found that participants were more likely to repeat another speaker's gestures if the original speaker gazed at the gestures. Moreover, speaker-gaze was a better predictor of this repetition than participants' own gaze. This supports the hypothesis that speakers' gaze at their gestures leads to covert attention shifts in observers, causing the gestures to be processed differently. Speaker-gaze could therefore be a valuable cue to the processing and production of gestures by artificial systems that interact with humans.

[1]  A. Garnham,et al.  The role of conversational hand gestures in a narrative task , 2007 .

[2]  Heather Shovelton,et al.  An experimental investigation of some properties of individual iconic gestures that mediate their communicative power. , 2002, British journal of psychology.

[3]  Iaira Boissevain,et al.  [Keeping an eye on it...]. , 2010, Tijdschrift voor diergeneeskunde.

[4]  Michael I. Posner,et al.  Structures and Functions of Selective Attention. , 1987 .

[5]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[6]  A. Hamilton,et al.  Eye contact enhances mimicry of intransitive hand movements , 2011, Biology Letters.

[7]  M. Posner,et al.  The attention system of the human brain. , 1990, Annual review of neuroscience.

[8]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[9]  Kenneth Holmqvist,et al.  Keeping an eye on gestures: Visual perception of gestures in face-to-face communication , 1999 .

[10]  Irene Kimbara.  On gestural mimicry , 2006 .

[11]  Sotaro Kita,et al.  Attention to Speech-Accompanying Gestures: Eye Movements and Information Uptake , 2009, Journal of nonverbal behavior.

[12]  Peter Wittenburg,et al.  ELAN: a Professional Framework for Multimodality Research , 2006, LREC.

[13]  D. McNeill,et al.  Speech-gesture mismatches: Evidence for one underlying representation of linguistic and nonlinguistic information , 1998 .

[14]  Heather Shovelton,et al.  Mapping the Range of Information Contained in the Iconic Hand Gestures that Accompany Spontaneous Speech , 1999 .

[15]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[16]  Irene Kimbara.  Gesture Form Convergence in Joint Description , 2008 .

[17]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[18]  G. Beattie,et al.  The Fixation and Processing of the Iconic Gestures That Accompany Talk , 2010 .

[19]  Stefan Kopp,et al.  Automatic and strategic alignment of co-verbal gestures in dialogue , 2013 .

[20]  M. Swerts,et al.  Adaptation in Gesture: Converging Hands or Converging Minds?. , 2012 .

[21]  M. Alibali,et al.  Effects of Visibility between Speaker and Listener on Gesture Production: Some Gestures Are Meant to Be Seen , 2001 .

[22]  J. Streeck.  Gesture as communication I: Its coordination with gaze and speech , 1993 .

[23]  Fey Parrill,et al.  Seeing and Hearing Double: The Influence of Mimicry in Speech and Gesture on Observers , 2006 .

[24]  J. Holler,et al.  Co-Speech Gesture Mimicry in the Process of Collaborative Referring During Face-to-Face Dialogue , 2011 .

[25]  Asli Ozyurek.  Do Speakers Design Their Cospeech Gestures for Their Addressees? The Effects of Addressee Location on Representational Gestures , 2002 .

[26]  Emiel Krahmer,et al.  The communicative import of gestures: evidence from a comparative analysis of human-human and human-machine interactions , 2007, AVSP.

[27]  D. McNeill.  Gesture and Thought , 2005 .

[28]  T. Chartrand,et al.  Self-monitoring without awareness: using mimicry as a nonconscious affiliation strategy. , 2003, Journal of personality and social psychology.

[29]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[30]  GEOFFREY BEATTIE,et al.  Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation , 1999 .

[31]  Emiel Krahmer,et al.  Seeing and Being Seen: The Effects on Gesture Production , 2011, J. Comput. Mediat. Commun..

[32]  Janet Beavin Bavelas,et al.  Gesturing on the telephone: Independent effects of dialogue and visibility. , 2008 .

[33]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[34]  Kenneth Holmqvist,et al.  What speakers do and what addressees look at: visual attention to gestures in human interaction live and on video , 2006 .

[35]  C. Goodwin.  Conversational Organization: Interaction Between Speakers and Hearers , 1981 .