A Virtual Agent as Vocabulary Trainer: Iconic Gestures Help to Improve Learners' Memory Performance

An important and often laborious task in foreign language acquisition is vocabulary learning. Research has repeatedly demonstrated that performing iconic gestures together with novel words has a beneficial effect on learning performance. Can these findings be transferred onto virtual agents applied in gesture-supported vocabulary training? We present a study investigating whether iconic gestures performed by a virtual agent and imitated by learners have an impact on verbal memory for words in a foreign language. In a within-subject design we compared participants’ memory performance achieved with the help of a virtual agent and those achieved with the help of a human trainer regarding both short-term learning effects and long-term decay effects. The overall results demonstrate improved memory scores when participants learned with a virtual agent. Especially high performers could profit from gesture-supported training with a virtual agent.

[1]  Michael Kipp,et al.  The Persona Zero-Effect: Evaluating Virtual Character Benefits on a Learning Task with Repeated Interactions , 2010, IVA.

[2]  L. Allen.  The Effects of Emblematic Gestures on the Development and Access of Mental Representations of French Expressions , 1995 .

[3]  Katharina von Kriegstein,et al.  Neural mechanisms of human communication , 2011 .

[4]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[5]  S. Kelly,et al.  Please Scroll down for Article Language and Cognitive Processes Brief Training with Co-speech Gesture Lends a Hand to Word Learning in a Foreign Language , 2022 .

[6]  Elisabeth André,et al.  The Persona Effect: How Substantial Is It? , 1998, BCS HCI.

[7]  R. Bollet,et al.  Personalizing E-Learning , 2002 .

[8]  G. Bente,et al.  Personalizing e-Learning. The Social Effects of Pedagogical Agents , 2010 .

[9]  M. Tellier.  The effect of gestures on second language memorisation by young children , 2008 .

[10]  Robbert-Jan Beun,et al.  Embodied Conversational Agents: Effects on Memory Performance and Anthropomorphisation , 2003, IVA.

[11]  Stefan Kopp,et al.  MURML: A Multimodal Utterance Representation Markup Language for Conversational Agents , 2002 .

[12]  K. Kriegstein,et al.  Gestures Enhance Foreign Language Learning , 2012, Biolinguistics.

[13]  Angela D. Friederici,et al.  Neural Correlates of High Performance in Foreign Language Vocabulary Learning. , 2010 .

[14]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[15]  Thomas R. Knösche,et al.  Body in mind: How gestures empower foreign language learning , 2011 .

[16]  Maria Virvou,et al.  Evaluating the persona effect of an interface agent in a tutoring system , 2002, J. Comput. Assist. Learn..

[17]  Soyoung Kim,et al.  Designing nonverbal communication for pedagogical agents: When less is more , 2009, Comput. Hum. Behav..