Toward a more embedded/extended perspective on the cognitive function of gestures

Gestures are often considered to be demonstrative of the embodied nature of the mind (Hostetter and Alibali, 2008). In this article, we review current theories and research targeted at the intra-cognitive role of gestures. We ask the question how can gestures support internal cognitive processes of the gesturer? We suggest that extant theories are in a sense disembodied, because they focus solely on embodiment in terms of the sensorimotor neural precursors of gestures. As a result, current theories on the intra-cognitive role of gestures are lacking in explanatory scope to address how gestures-as-bodily-acts fulfill a cognitive function. On the basis of recent theoretical appeals that focus on the possibly embedded/extended cognitive role of gestures (Clark, 2013), we suggest that gestures are external physical tools of the cognitive system that replace and support otherwise solely internal cognitive processes. That is gestures provide the cognitive system with a stable external physical and visual presence that can provide means to think with. We show that there is a considerable amount of overlap between the way the human cognitive system has been found to use its environment, and how gestures are used during cognitive processes. Lastly, we provide several suggestions of how to investigate the embedded/extended perspective of the cognitive function of gestures.

[1]  T. Trabasso,et al.  Offering a Hand to Pragmatic Understanding: The Role of Speech and Gesture in Comprehension and Memory , 1999 .

[2]  Gary L. Allen,et al.  Gestures Accompanying Verbal Route Directions: Do They Point to a New Avenue for Examining Spatial Representations? , 2003, Spatial Cogn. Comput..

[3]  Fred Paas,et al.  Effects of imitating gestures during encoding or during retrieval of novel verbs on children's test performance. , 2013, Acta psychologica.

[4]  A. Baddeley.  Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[5]  Martha W. Alibali,et al.  Language, gesture, action! A test of the Gesture as Simulated Action framework , 2010 .

[6]  Michael P. Kaschak,et al.  Perception of Auditory Motion Affects Language Processing , 2006, Cogn. Sci..

[7]  Bernard Rimé,et al.  Effects of relative immobilization on the speaker's nonverbal behavior and on the dialogue imagery level , 1984 .

[8]  S. Kita,et al.  Spontaneous gestures during mental rotation tasks: insights into the microdevelopment of the motor strategy. , 2008, Journal of experimental psychology. General.

[9]  Mary Hegarty,et al.  The Role of Gestures in Mental Animation , 2005, Spatial Cogn. Comput..

[10]  David Kirsh,et al.  Thinking with external representations , 2010, AI & SOCIETY.

[11]  K. A. Ericsson,et al.  Skill and Working Memory , 1982 .

[12]  Henrik Svensson,et al.  Embodied simulation as off-line representation , 2007 .

[13]  Friedemann Pulvermüller,et al.  Reading cinnamon activates olfactory brain regions , 2006, NeuroImage.

[14]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[15]  Daniel L. Schwartz,et al.  Physically Distributed Learning: Adapting and Reinterpreting Physical Environments in the Development of Fraction Concepts , 2005, Cogn. Sci..

[16]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[17]  M. Wheeler.  Reconstructing the Cognitive World: The Next Step , 2005 .

[18]  A. Anastasi.  Individual differences. , 2020, Annual review of psychology.

[19]  Akira Takada,et al.  A Prelinguistic Gestural Universal of Human Communication , 2012, Cogn. Sci..

[20]  K. Vogeley,et al.  Toward a second-person neuroscience 1 , 2013, Behavioral and Brain Sciences.

[21]  A. Chemero.  Radical Embodied Cognitive Science , 2009 .

[22]  Alan Kingstone,et al.  Rotating With Rotated Text: A Natural Behavior Approach to Investigating Cognitive Offloading , 2014, Cogn. Sci..

[23]  Martha W. Alibali,et al.  Gesture During Mental Rotation , 2011, CogSci.

[24]  Doreen Meier,et al.  Cognitive Systems And The Extended Mind , 2016 .

[25]  D. McNeill.  So you think gestures are nonverbal , 1985 .

[26]  Wai-Tat Fu,et al.  A Dynamic Context Model of Interactive Behavior , 2011, Cogn. Sci..

[27]  Damian G. Stephen,et al.  Dynamics of representational change: entropy, action, and cognition. , 2009, Journal of experimental psychology. Human perception and performance.

[28]  P. Zelazo.  The Oxford Handbook of Developmental Psychology , 2013 .

[29]  E. D. Paolo,et al.  Can social interaction constitute social cognition? , 2010, Trends in Cognitive Sciences.

[30]  Sotaro Kita,et al.  What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking , 2003 .

[31]  M. Hegarty,et al.  Representational Translation With Concrete Models in Organic Chemistry , 2012 .

[32]  Karen Salmon,et al.  Giving Memory a Hand: Instructing Children to Gesture Enhances their Event Recall , 2005 .

[33]  A. Clark,et al.  The Extended Mind , 1998, Analysis.

[34]  E. D. Paolo,et al.  Participatory sense-making , 2007 .

[35]  Gesture when there is no visual model , 1998 .

[36]  Susan M. Wagner,et al.  Probing the Mental Representation of Gesture: Is Handwaving Spatial?. , 2004 .

[37]  M. Tomasello,et al.  A new look at infant pointing. , 2007, Child development.

[38]  S. Gallagher.  How the body shapes the mind , 2005 .

[39]  R. Krauss.  Why Do We Gesture When We Speak? , 1998 .

[40]  J. Gómez,et al.  Pointing gestures as a cognitive tool in young children: experimental evidence. , 2011, Journal of experimental child psychology.

[41]  F. Adams,et al.  The bounds of cognition , 2001 .

[42]  J. Gómez,et al.  Private Pointing and private speech: developing parallelisms , 2009 .

[43]  Susan Goldin-Meadow,et al.  Gesture in Thought , 2012 .

[44]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[45]  Guy Dove.  On the need for Embodied and Dis-Embodied Cognition , 2011, Front. Psychology.

[46]  E. Hutchins.  Material anchors for conceptual blends , 2005 .

[47]  Ezequiel Morsella,et al.  Muscular Activity in the Arm during Lexical Retrieval: Implications for Gesture-Speech Theories , 2005, Journal of psycholinguistic research.

[48]  Martha W. Alibali,et al.  Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information , 2005, Spatial Cogn. Comput..

[49]  M. Alibali,et al.  Effects of Visibility between Speaker and Listener on Gesture Production: Some Gestures Are Meant to Be Seen , 2001 .

[50]  Susan Goldin-Meadow,et al.  Gesture’s Role in Learning and Development , 2013 .

[51]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[52]  Rebecca Boncoddo,et al.  The emergence of a novel representation from action: evidence from preschoolers. , 2010, Developmental science.

[53]  Robert M. Krauss,et al.  Gesture and Speech in Spontaneous and Rehearsed Narratives , 1994 .

[54]  A. Clark,et al.  Supersizing The Mind Embodiment Action And Cognitive Extension Andy Clark , 2016 .

[55]  S T Boysen,et al.  Indicating acts during counting by a chimpanzee (Pan troglodytes). , 1995, Journal of comparative psychology.

[56]  Emiel Krahmer,et al.  Does our speech change when we cannot gesture? , 2014, Speech Commun..

[57]  M. Wheeler.  Is Cognition Embedded or Extended? The Case of Gestures , 2013 .

[58]  S. Kita,et al.  The nature of gestures ' beneficial role in spatial problem solving , 2013 .

[59]  A. Kendon.  Do Gestures Communicate? A Review , 1994 .

[60]  F. Pulvermüller,et al.  Reading salt activates gustatory brain regions: fMRI evidence for semantic grounding in a novel sensory modality. , 2012, Cerebral cortex.

[61]  Michael K. Tanenhaus,et al.  Embodied communication: Speakers’ gestures affect listeners’ actions , 2009, Cognition.

[62]  Wai-Tat Fu,et al.  Soft constraints in interactive behavior: the case of ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head , 2004, Cogn. Sci..

[63]  Edward C. Warburton,et al.  The Cognitive Benefits of Movement Reduction , 2013, Psychological science.

[64]  Ezequiel Morsella,et al.  The role of gestures in spatial working memory and speech. , 2004, The American journal of psychology.

[65]  Susan M. Wagner,et al.  Explaining Math: Gesturing Lightens the Load , 2001, Psychological science.

[66]  R Wesp,et al.  Gestures maintain spatial imagery. , 2001, The American journal of psychology.

[67]  A. Glenberg,et al.  Grounding language in bodily states: The case for emotion , 2005 .

[68]  Giyoo Hatano,et al.  Digit memory of grand experts in abacus-derived mental calculation , 1983, Cognition.

[69]  T. Graham,et al.  The role of gesture in children's learning to count. , 1999, Journal of experimental child psychology.

[70]  Elizabeth Bates,et al.  The Acquisition of Performatives Prior to Speech. , 1975 .

[71]  S. Goldin-Meadow,et al.  Hearing Gesture: How Our Hands Help Us Think , 2003 .

[72]  Michael L. Anderson,et al.  Eroding the boundaries of cognition: implications of embodiment(1). , 2012, Topics in cognitive science.

[73]  Gale Stam.  Gesture's Role in Learning , 2013 .

[74]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[75]  J. Holler,et al.  The Effect of Common Ground on How Speakers Use Gesture and Speech to Represent Size Information , 2007 .

[76]  Fred Paas,et al.  Words in action: Using gestures to improve verb learning in primary school children , 2014 .

[77]  Rajesh P. N. Rao,et al.  Embodiment is the foundation, not a level , 1996, Behavioral and Brain Sciences.

[78]  Autumn B. Hostetter,et al.  Visible embodiment: Gestures as simulated action , 2008, Psychonomic bulletin & review.

[79]  M. Tomasello.  Origins of human communication , 2008 .

[80]  Martha W. Alibali,et al.  The Role of Gesture in Children's Comprehension of Spoken Language:Now They Need It, Now They Don't , 2000 .

[81]  Lars Marstaller,et al.  Individual differences in the gesture effect on working memory , 2013, Psychonomic bulletin & review.

[82]  F. Pulvermüller,et al.  Motor cognition–motor semantics: Action perception theory of cognition and communication , 2014, Neuropsychologia.

[83]  Sotaro Kita,et al.  Spontaneous Gestures Influence Strategy Choices in Problem Solving , 2011, Psychological science.

[84]  D. Schwartz,et al.  Inferences through imagined actions: Knowing by simulated doing. , 1999 .

[85]  D. Ballard,et al.  Memory Representations in Natural Tasks , 1995, Journal of Cognitive Neuroscience.

[86]  R. Klatzky,et al.  Teachers’ gestures facilitate students’ learning: A lesson in symmetry , 2003 .

[87]  M. Alibali,et al.  The function of gesture in learning to count: more than keeping track * , 1999 .

[88]  R. A. Carlson,et al.  External support and the development of problem-solving routines. , 1999 .

[89]  W. Pouw,et al.  An Embedded and Embodied Cognition Review of Instructional Manipulatives , 2014 .

[90]  Fred Paas,et al.  Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning , 2013, Comput. Hum. Behav..

[91]  R. Krauss,et al.  The Role of Speech-Related Arm/Hand Gestures in Word Retrieval , 2001 .

[92]  S. Kita,et al.  Individual Differences in Frequency and Saliency of Speech-Accompanying Gestures: The Role of Cognitive Abilities and Empathy , 2013, Journal of experimental psychology. General.

[93]  Paul P. Maglio,et al.  On Distinguishing Epistemic from Pragmatic Action , 1994, Cogn. Sci..

[94]  S. Kelly,et al.  Broadening the units of analysis in communication: speech and nonverbal behaviours in pragmatic comprehension , 2001, Journal of Child Language.

[95]  Giyoo Hatano,et al.  Performance of expert abacus operators , 1977, Cognition.

[96]  R. Krauss,et al.  PSYCHOLOGICAL SCIENCE Research Article GESTURE, SPEECH, AND LEXICAL ACCESS: The Role of Lexical Movements in Speech Production , 2022 .

[97]  Rolf A. Zwaan.  Embodiment and language comprehension: reframing the discussion , 2014, Trends in Cognitive Sciences.

[98]  David Kirsh.  Interaction, External Representation and Sense Making , 2009 .

[99]  J. van der Steen,et al.  Copying Strategies for Patterns by Children and Adults , 2000, Perceptual and motor skills.

[100]  L. Barsalou.  Grounded cognition. , 2008, Annual review of psychology.

[101]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[102]  M. Arbib,et al.  Language is handy but is it embodied? , 2014, Neuropsychologia.

[103]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[104]  Edwin Hutchins,et al.  How a Cockpit Remembers Its Speeds , 1995, Cogn. Sci..

[105]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[106]  E. Bredo.  Reconstructing educational psychology: Situated cognition and Deweyian pragmatism , 1994 .

[107]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[108]  Shaun Gallagher,et al.  Philosophical Antecedents of Situated Cognition , 2008 .

[109]  E. Nicoladis,et al.  Lending a Hand to Imagery? The Impact of Visuospatial Working Memory Interference Upon Iconic Gesture Production in a Narrative Task , 2014 .

[110]  Yihsiu Chen,et al.  Language and Gesture: Lexical gestures and lexical access: a process model , 2000 .

[111]  M. Argyle,et al.  A Cross-Cultural Study of the Communication of Extra-Verbal Meaning by Gesture , 1975 .

[112]  Susan Goldin-Meadow,et al.  Gesturing Saves Cognitive Resources When Talking About Nonpresent Objects , 2010, Cogn. Sci..

[113]  Monika Richter,et al.  Cognition In The Wild , 2016 .

[114]  Zdravko Radman,et al.  The hand, an organ of the mind : what the manual tells the mental , 2013 .

[115]  Susan Goldin-Meadow,et al.  Action’s Influence on Thought: The Case of Gesture , 2010, Perspectives on psychological science : a journal of the Association for Psychological Science.

[116]  Steve Benford,et al.  The role of physical representations in solving number problems: A comparison of young children's use of physical and virtual materials , 2010, Comput. Educ..

[117]  Sotaro Kita,et al.  How representational gestures help speaking , 2000 .

[118]  E. Thelen,et al.  Hand, mouth and brain. The dynamic emergence of speech and gesture , 1999 .

[119]  A. Clark.  Supersizing the Mind , 2008 .

[120]  Susan Goldin-Meadow,et al.  Gestures, but not meaningless movements, lighten working memory load when explaining math , 2012, Language and cognitive processes.