Understanding gesture: is the listener's motor system involved?

Listeners are able to glean information from the gestures that speakers produce, seemingly without conscious awareness. However, little is known about the mechanisms that underlie this process. Research on human action understanding shows that perceiving another's actions results in automatic activation of the motor system in the observer, which then affects the observer's understanding of the actor's goals. We ask here whether perceiving another's gesture can similarly result in automatic activation of the motor system in the observer. In Experiment 1, we first established a new procedure in which listener response times are used to study how gesture impacts sentence comprehension. In Experiment 2, we used this procedure, in conjunction with a secondary motor task, to investigate whether the listener's motor system is involved in this process. We showed that moving arms and hands (but not legs and feet) interferes with the listener's ability to use information conveyed in a speaker's hand gestures. Our data thus suggest that understanding gesture relies, at least in part, on the listener's own motor system.

[1]  Sian L. Beilock,et al.  Embodied Preference Judgments , 2007, Psychological science.

[2]  R. Passingham,et al.  Action observation and acquired motor skills: an FMRI study with expert dancers. , 2005, Cerebral cortex.

[3]  Uta Frith,et al.  Your Own Action Influences How You Perceive Another Person's Action , 2004, Current Biology.

[4]  Susan Goldin-Meadow,et al.  Transitional knowledge in the acquisition of concepts , 1988 .

[5]  Sarah A. Gerson,et al.  The emergence of intention attribution in infancy. , 2009, The psychology of learning and motivation.

[6]  Rolf A. Zwaan,et al.  Language Comprehenders Mentally Represent the Shapes of Objects , 2002, Psychological science.

[7]  Wolfgang Prinz,et al.  Twin Peaks: An ERP Study of Action Planning and Control in Coacting Individuals , 2006, Journal of Cognitive Neuroscience.

[8]  Susan Goldin-Meadow,et al.  thoughts to ordinary listeners , 1999 .

[9]  W. Prinz,et al.  Motor learning enhances perceptual judgment: a case for action-perception transfer , 2001, Psychological research.

[10]  Rolf A. Zwaan,et al.  Seeing, acting, understanding: motor resonance in language comprehension. , 2006, Journal of experimental psychology. General.

[11]  Winona Snapp-Childs,et al.  Perceptual learning immediately yields new stable motor coordination. , 2010, Journal of experimental psychology. Human perception and performance.

[12]  Bernhard Hommel,et al.  How Social Are Task Representations? , 2009, Psychological science.

[13]  G P Bingham,et al.  Visual perception of mean relative phase and phase variability. , 2000, Journal of experimental psychology. Human perception and performance.

[14]  Susan Goldin-Meadow,et al.  Action’s Influence on Thought: The Case of Gesture , 2010, Perspectives on psychological science : a journal of the Association for Psychological Science.

[15]  Sian L. Beilock,et al.  Gesture Changes Thought by Grounding It in Action , 2010, Psychological science.

[16]  S. Goldin-Meadow,et al.  The mismatch between gesture and speech as an index of transitional knowledge , 1986, Cognition.

[17]  Martha W. Alibali,et al.  Language, gesture, action! A test of the Gesture as Simulated Action framework , 2010 .

[18]  J. Mazziotta,et al.  Transcranial magnetic stimulation studies of the human mirror neuron system , 2002 .

[19]  Günther Knoblich,et al.  The Social Nature of Perception and Action , 2006 .

[20]  S Goldin-Meadow,et al.  Comprehension and production of gesture in combination with speech in one-word speakers , 1992, Journal of Child Language.

[21]  K. Pine,et al.  More gestures than answers: children learning about balance. , 2004, Developmental psychology.

[22]  Sian L. Beilock,et al.  Expertise and its embodiment: Examining the impact of sensorimotor skill expertise on the representation of action-related text , 2006, Psychonomic bulletin & review.

[23]  Michael K. Tanenhaus,et al.  Embodied communication: Speakers’ gestures affect listeners’ actions , 2009, Cognition.

[24]  S. Goldin-Meadow,et al.  Hearing Gesture: How Our Hands Help Us Think , 2003 .

[25]  Justine Cassell,et al.  Communicative Effects of Speech-Mismatched Gestures , 1994 .

[26]  Sian L. Beilock,et al.  Embodied memory judgments: a case of motor fluency. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[27]  Autumn B. Hostetter,et al.  Visible embodiment: Gestures as simulated action , 2008, Psychonomic bulletin & review.

[28]  S. Kelly,et al.  Broadening the units of analysis in communication: speech and nonverbal behaviours in pragmatic comprehension , 2001, Journal of Child Language.

[29]  Catherine L Reed,et al.  Action during body perception: Processing time affects self–other correspondences , 2007, Social neuroscience.

[30]  G. Rizzolatti,et al.  Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study , 2001, The European journal of neuroscience.

[31]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[32]  G P Bingham,et al.  Visual perception of the relative phasing of human limb movements , 1999, Perception & psychophysics.

[33]  W. Prinz,et al.  Representing others' actions: just like one's own? , 2003, Cognition.

[34]  Maggie Shiffrar,et al.  Walking perception by walking observers. , 2005, Journal of experimental psychology. Human perception and performance.

[35]  R. Ellis,et al.  Action priming by briefly presented objects. , 2004, Acta psychologica.

[36]  David R. Collins,et al.  Perceptual coupling in rhythmic movement coordination: stable perception leads to stable action , 2005, Experimental Brain Research.

[37]  Sian L. Beilock,et al.  Sports experience changes the neural processing of action language , 2008, Proceedings of the National Academy of Sciences.

[38]  Rolf A. Zwaan,et al.  PSYCHOLOGICAL SCIENCE Research Article THE EFFECT OF IMPLIED ORIENTATION DERIVED FROM VERBAL CONTEXT ON PICTURE RECOGNITION , 2022 .

[39]  H. Bekkering,et al.  Joint action: bodies and minds moving together , 2006, Trends in Cognitive Sciences.

[40]  M. Farah,et al.  The psychological reality of the body schema: a test with normal participants. , 1995, Journal of experimental psychology. Human perception and performance.

[41]  Susan Goldin-Meadow,et al.  Assessing knowledge conveyed in gesture : Do teachers have the upper hand ? , 1997 .