The Relation Between Cognitive Abilities and the Distribution of Semantic Features Across Speech and Gesture in 4-year-olds

When young children learn to use language, they start to use their hands in co-verbal gesturing. There are, however, considerable differences between children, and it is not completely understood what these individual differences are due to. We studied how children at 4 years of age employ speech and iconic gestures to convey meaning in different kinds of spatial event descriptions, and how this relates to their cognitive abilities. Focusing on spontaneous illustrations of actions, we applied a semantic feature (SF) analysis to characterize combinations of speech and gesture meaning and related them to the child's visual-spatial abilities or abstract/concrete reasoning abilities (measured using the standardized SON-R 2 1 2 - 7 test). Results show that children with higher cognitive abilities convey significantly more meaning via gesture and less solely via speech. These findings suggest that young children's use of cospeech representational gesturing is positively related to their mental representation and reasoning abilities.

[1]  Susan Goldin-Meadow,et al.  Gesturing has a larger impact on problem-solving than action, even when action is accompanied by words , 2015, Language, cognition and neuroscience.

[2]  Stefan Kopp,et al.  MODELING THE PRODUCTION OF COVERBAL ICONIC GESTURES BY LEARNING BAYESIAN DECISION NETWORKS , 2010, Appl. Artif. Intell..

[3]  Stefan Kopp,et al.  Gesture and speech in interaction: An overview , 2014, Speech Commun..

[4]  Can gesticulation help aphasic people speak, or rather, communicate? , 2006 .

[5]  Martha W. Alibali,et al.  Cognitive skills and gesture–speech redundancy: Formulation difficulty or communicative strategy? , 2011 .

[6]  B. Butterworth,et al.  Iconic gestures, imagery, and word retrieval in speech , 1997 .

[7]  Janet Beavin Bavelas,et al.  An experimental study of when and how speakers use gestures to communicate , 2002 .

[8]  E. Franz,et al.  On the relationship between fluid intelligence, gesture production, and brain structure☆ , 2010 .

[9]  Autumn B. Hostetter,et al.  Gesture-speech integration in children with specific language impairment. , 2014, International journal of language & communication disorders.

[10]  Autumn B Hostetter,et al.  Gesture-speech integration in narrative: Are children less redundant than adults? , 2009, Gesture.

[11]  M. Alibali,et al.  Gesture-Speech Mismatch and Mechanisms of Learning: What the Hands Reveal about a Child′s State of Mind , 1993, Cognitive Psychology.

[12]  Cornelia Müller,et al.  128. Gestural modes of representation as techniques of depiction , 2014 .

[13]  Sotaro Kita,et al.  Conceptualisation load triggers gesture production , 2007 .

[14]  E A Franz,et al.  Bimanual gestures: Expressions of spatial representations that accompany speech processes , 2005, Laterality.

[15]  Sotaro Kita,et al.  Development of cross-linguistic variation in speech and gesture: motion events in English and Turkish. , 2008, Developmental psychology.

[16]  S. Goldin-Meadow,et al.  The mismatch between gesture and speech as an index of transitional knowledge , 1986, Cognition.

[17]  T. Göksun,et al.  Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources , 2020, Frontiers in Psychology.

[18]  M. Alibali,et al.  Gesture and the process of speech production: We think, therefore we gesture , 2000 .

[19]  Susan Goldin-Meadow,et al.  Gesture Paves the Way for Language Development , 2005, Psychological science.

[20]  E. Nicoladis,et al.  Gesturers Tell a Story Creatively; Non-Gesturers Tell it like it Happened , 2020 .

[21]  Martha W. Alibali,et al.  I see it in my hands’ eye: Representational gestures reflect conceptual demands , 2007 .

[22]  S. Kita,et al.  Individual Differences in Frequency and Saliency of Speech-Accompanying Gestures: The Role of Cognitive Abilities and Empathy , 2013, Journal of experimental psychology. General.

[23]  D. Slobin.  Thinking for Speaking , 1987 .

[24]  Isabell Wartenburger,et al.  Show your hands — Are you really clever? Reasoning, gesture production, and intelligence , 2011 .

[25]  S. Goldin-Meadow,et al.  The Role of Gesture in Learning: Do Children Use Their Hands to Change Their Minds? , 2006 .

[26]  Martha W. Alibali,et al.  Raise your hand if you’re spatial: Relations between verbal and spatial skills and gesture production , 2007 .

[27]  Fey Parrill,et al.  Effects of input modality on speech-gesture integration , 2010 .

[28]  Tilbe Göksun,et al.  Naming and gesturing spatial relations: Evidence from focal brain-injured individuals , 2013, Neuropsychologia.

[29]  Hao Yan,et al.  Coordination and context-dependence in the generation of embodied conversation , 2000, INLG.

[30]  M. Spinelli,et al.  Gesture production in the narratives of preschool children , 2016 .

[31]  Steven M. Weisberg,et al.  When gestures show us the way: Co-thought gestures selectively facilitate navigation and spatial memory , 2018, Spatial Cogn. Comput..

[32]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[33]  K. Rohlfing,et al.  Children’s Narrative Elaboration After Reading a Storybook Versus Viewing a Video , 2020, Frontiers in Psychology.

[34]  Kim A. Kastens,et al.  Gestures in Geology: The Roles of Spatial Skills, Expertise, and Communicative Context , 2010, Spatial Cognition.

[35]  Stefan Kopp,et al.  Modeling the Semantic Coordination of Speech and Gesture under Cognitive and Linguistic Constraints , 2013, IVA.

[36]  Christina Y. Tzeng,et al.  Using and Seeing Co-speech Gesture in a Spatial Task , 2015 .

[37]  Judith Holler,et al.  How iconic gestures and speech interact in the representation of meaning: Are both aspects really integral to the process? , 2003 .

[38]  Susan Bell Trickett,et al.  The Relationship Between Spatial Transformations and Iconic Gestures , 2006, Spatial Cogn. Comput..

[39]  Katharina J. Rohlfing.  Learning Language from the Use of Gestures , 2019 .

[40]  Thomas F. Shipley,et al.  Individual differences in mental rotation: what does gesture tell us? , 2013, Cognitive Processing.

[41]  Jan Peter de Ruiter,et al.  The Interplay Between Gesture and Speech in the Production of Referring Expressions: Investigating the Tradeoff Hypothesis , 2012, Top. Cogn. Sci..

[42]  Nina C Capone,et al.  Gesture development: a review for clinical and research practices. , 2004, Journal of speech, language, and hearing research : JSLHR.

[43]  Prisca Stenneken,et al.  Children's lexical skills and task demands affect gestural behavior in mothers of late-talking children and children with typical language development [Reprint] , 2010 .

[44]  Susan Goldin-Meadow,et al.  Gesturing Saves Cognitive Resources When Talking About Nonpresent Objects , 2010, Cogn. Sci..

[45]  S. Kita,et al.  The Development of the Ability to Semantically Integrate Information in Speech and Iconic Gesture in Comprehension , 2015, Cogn. Sci..

[46]  Sotaro Kita,et al.  How Do Gestures Influence Thinking and Speaking? The Gesture-for-Conceptualization Hypothesis , 2017, Psychological review.

[47]  S. Kita,et al.  Seeing Iconic Gestures While Encoding Events Facilitates Children's Memory of These Events. , 2019, Child development.

[48]  Karen Emmorey,et al.  Gesture Helps, Only If You Need It: Inhibiting Gesture Reduces Tip-of-the-Tongue Resolution for Those With Weak Short-Term Memory , 2021, Cogn. Sci..

[49]  Stacy B. Ehrlich,et al.  The importance of gesture in children's spatial reasoning. , 2006, Developmental psychology.

[50]  Jean-Marc Colletta,et al.  Age-related changes in co-speech gesture and narrative: Evidence from French children and adults , 2010, Speech Commun..

[51]  Autumn B. Hostetter,et al.  When do gestures communicate? A meta-analysis. , 2011, Psychological bulletin.

[52]  Susan Goldin-Meadow,et al.  When gesture-speech combinations do and do not index linguistic change , 2009, Language and cognitive processes.

[53]  Susan M. Wagner,et al.  Explaining Math: Gesturing Lightens the Load , 2001, Psychological science.

[54]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.