I see it in my hands’ eye: Representational gestures reflect conceptual demands

The Information Packaging Hypothesis (Kita, 2000) holds that gestures play a role in conceptualising information for speaking. According to this view, speakers will gesture more when describing difficult-to-conceptualise information than when describing easy-to-conceptualise information. In the present study, 24 participants described ambiguous dot patterns under two conditions. In the dots-plus-shapes condition, geometric shapes connected the dots, and participants described the patterns in terms of those shapes. In the dots-only condition, no shapes were present, and participants generated their own geometric conceptualisations and described the patterns. Participants gestured at a higher rate in the dots-only condition than in the dots-plus-shapes condition. The results support the Information Packaging Hypothesis and suggest that gestures occur when information is difficult to conceptualise.

[1]  Michael Wilson,et al.  MRC psycholinguistic database: Machine-usable dictionary, version 2.00 , 1988 .

[2]  Uri Hadar,et al.  Two Types of Gesture and Their Role in Speech Production , 1989 .

[3]  Yihsiu Chen,et al.  Language and Gesture: Lexical gestures and lexical access: a process model , 2000 .

[4]  M. Jeannerod Simulation of action as a unifying concept for motor cognition. , 2003 .

[5]  S. Kosslyn Mental images and the Brain , 2005, Cognitive neuropsychology.

[6]  Scott H. Johnson-Frey Taking action : cognitive neuroscience perspectives on intentional acts , 2003 .

[7]  A. Glenberg,et al.  What memory is for: Creating meaning in the service of action , 1997, Behavioral and Brain Sciences.

[8]  M. Peterson,et al.  Mental images can be ambiguous: Reconstruals and reference-frame reversals , 1992, Memory & cognition.

[9]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[10]  Michael Wilson MRC Psycholinguistic Database , 2001 .

[11]  S. Goldin-Meadow,et al.  The mismatch between gesture and speech as an index of transitional knowledge , 1986, Cognition.

[12]  M. Alibali,et al.  Effects of Visibility between Speaker and Listener on Gesture Production: Some Gestures Are Meant to Be Seen , 2001 .

[13]  Sotaro Kita,et al.  Conceptualisation load triggers gesture production , 2007 .

[14]  Wolff-Michael Roth,et al.  From action to discourse: The bridging function of gestures , 2002, Cognitive Systems Research.

[15]  Frieda Goldman-Eisler,et al.  Speech Production and the Predictability of Words in Context , 1958 .

[16]  Brian Butterworth,et al.  Gesture and Silence as Indicators of Planning in Speech , 1978 .

[17]  S. Kosslyn,et al.  Visual mental images can be ambiguous: insights from individual differences in spatial transformation abilities , 2002, Cognition.

[18]  A. Cohen,et al.  Intentionality in the use of hand illustrators in face-to-face communication situations. , 1973 .

[19]  S. Schachter,et al.  Speech Disfluency and the Structure of Knowledge , 1991 .

[20]  Daniel L. Schwartz,et al.  Inferences through imagined actions: Knowing by simulated doing. , 1999 .

[21]  C. Creider Hand and Mind: What Gestures Reveal about Thought , 1994 .

[22]  Willem J. M. Levelt,et al.  A theory of lexical access in speech production , 1999, Behavioral and Brain Sciences.

[23]  Susan M. Wagner,et al.  Explaining Math: Gesturing Lightens the Load , 2001, Psychological science.

[24]  Cesare Cornoldi,et al.  Visuo-spatial working memory and individual differences , 2003 .

[25]  Robert M. Krauss,et al.  Gesture and Speech in Spontaneous and Rehearsed Narratives , 1994 .

[26]  R Wesp,et al.  Gestures maintain spatial imagery. , 2001, The American journal of psychology.

[27]  Susan Goldin-Meadow,et al.  Illuminating Mental Representations Through Speech and Gesture , 1999 .

[28]  Martha W. Alibali,et al.  Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information , 2005, Spatial Cogn. Comput..

[29]  Beth Levy,et al.  Conceptual Representations in Lan-guage Activity and Gesture , 1980 .

[30]  M. Alibali,et al.  Gesture and the process of speech production: We think, therefore we gesture , 2000 .

[31]  Sotaro Kita,et al.  How representational gestures help speaking , 2000 .

[32]  Martha W. Alibali,et al.  On the Tip of the Mind: Gesture as a Key to Conceptualization , 2004 .

[33]  Chapter 16 Imagery in mental construction and decomposition tasks , 1991 .

[34]  R. Krauss,et al.  PSYCHOLOGICAL SCIENCE Research Article GESTURE, SPEECH, AND LEXICAL ACCESS: The Role of Lexical Movements in Speech Production , 2022 .

[35]  J. D. Ruiter The production of gesture and speech , 2000 .

[36]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[37]  S. Kosslyn Image and Brain , 1994 .

[38]  Sotaro Kita,et al.  Does Gesture Help Processes of Speech Production? Evidence for Conceptual Level Facilitation , 2001 .

[39]  J. Bavelas,et al.  Gestures Specialized for Dialogue , 1995 .

[40]  Matthew Flatt,et al.  PsyScope: An interactive graphic system for designing and controlling experiments in the psychology laboratory using Macintosh computers , 1993 .

[41]  P. Tannenbaum,et al.  Word predictability in the environments of hesitations , 1965 .

[42]  Ezequiel Morsella,et al.  The role of gestures in spatial working memory and speech. , 2004, The American journal of psychology.

[43]  H. H. Clark,et al.  Using uh and um in spontaneous speaking , 2002, Cognition.

[44]  S. Kosslyn Image and Brain: The Resolution of the Imagery Debate , 1994, Journal of Cognitive Neuroscience.

[45]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[46]  R. Krauss,et al.  The Communicative Value of Conversational Hand Gesture , 1995 .

[47]  Sotaro Kita,et al.  What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking , 2003 .

[48]  G. Beattie,et al.  What properties of talk are associated with the generation of spontaneous iconic hand gestures? , 2002, The British journal of social psychology.

[49]  Robert J. Hartsuiker,et al.  Maximal input and feedback in production and comprehension , 2005 .

[50]  Rick Grush,et al.  The emulation theory of representation: Motor control, imagery, and perception , 2004, Behavioral and Brain Sciences.