Keeping an eye on gestures: Visual perception of gestures in face-to-face communication

Since listeners usually look at the speaker's face, gestural information has to be absorbed through peripheral visual perception. In the literature, it has been suggested that listeners look at gestures under certain circumstances: 1) when the articulation of the gesture is peripheral; 2) when the speech channel is insufficient for comprehension; and 3) when the speaker him- or herself indicates that the gesture is worthy of attention. The research here reported employs eye tracking techniques to study the perception of gestures in face-to-face interaction. The improved control over the listener's visual channel allows us to test the validity of the above claims. We present preliminary findings substantiating claims 1 and 3, and relate them to theoretical proposals in the literature and to the issue of how visual and cognitive attention are related.

[1]  M I Posner,et al.  Neuroanatomy, circuitry and plasticity of word reading. , 1999, Neuroreport.

[2]  W. Rogers,et al.  THE CONTRIBUTION OF KINESIC ILLUSTRATORS TOWARD THE COMPREHENSION OF VERBAL BEHAVIOR WITHIN UTTERANCES , 1978 .

[3]  Adam Kendon,et al.  How gestures can become like words , 1988 .

[4]  M. Land.  Visual Perception: Physiology, Psychology and Ecology, Vicki Bruce, Patrick Green. Lawrence Erlbaum, London (1985), xiii, +369. Price £8.95 (paperback) , 1986 .

[5]  Andrew Hollingworth,et al.  Eye Movements During Scene Viewing: An Overview , 1998 .

[6]  Keziah Latham.  Psychophysical investigations of human peripheral vision. , 1995 .

[7]  Developmental Effects on the Reception of Signs in Peripheral Vision by Deaf Students , 2013 .

[8]  H. H. Clark,et al.  Quotations as Demonstrations. , 1990 .

[9]  C. Goodwin.  Gestures as a resource for the organization of mutual orientation , 1986 .

[10]  Scott K. Liddell.  American Sign Language Syntax , 1981 .

[11]  D. McNeill,et al.  Speech-gesture mismatches: Evidence for one underlying representation of linguistic and nonlinguistic information , 1998 .

[12]  M. Virginia Swisher.  The Reception of Signs in Peripheral Vision. , 1989 .

[13]  M. Posner,et al.  Attention and the detection of signals. , 1980, Journal of experimental psychology.

[14]  G. Beattie,et al.  Cross-cultural similarities in gestures: The deep relationship between gestures and speech which transcends language barriers , 1996 .

[15]  M. Knapp,et al.  The Interaction of Visual and Verbal Features in Human Communication , 1992 .

[16]  D. McNeill,et al.  Abstract deixis , 1993 .

[17]  Timo Haukioja.  Pointing in sign language and gesture: An alternative interpretation , 1993 .

[18]  Justine Cassell,et al.  Communicative Effects of Speech-Mismatched Gestures , 1994 .

[19]  K. Tuite.  The production of gesture , 1993 .

[20]  J Theeuwes,et al.  Visual selective attention: a theoretical analysis. , 1993, Acta psychologica.

[21]  R. E. Morrison,et al.  Manipulation of stimulus onset delay in reading: evidence for parallel programming of saccades. , 1984, Journal of experimental psychology. Human perception and performance.

[22]  J. Streeck.  Gesture as communication II: The audience as co-author. , 1994 .

[23]  J. Streeck.  Gesture as communication I: Its coordination with gaze and speech , 1993 .

[24]  M. Gullberg.  Gesture as a communication strategy in second language discourse , 1998 .

[25]  A. Kendon.  Do Gestures Communicate? A Review , 1994 .

[26]  Patricia Siple,et al.  Visual Constraints for Sign Language Communication , 2013 .

[27]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[28]  D. McNeill.  Hand and Mind , 1995 .