Digital learning activities delivered by eloquent instructor avatars: scaling with problem instance

We present an approach for achieving scalable authoring of digital learning activities, without sacrificing delivery eloquence. A computer animation character serves as an instructor avatar that not only speaks but also makes deictic, iconic and charisma gestures. The avatar is controlled via a text script, without the prerequisites of computer programming or animation expertise. Given a script for a problem, the system automatically generates scripts for additional instances of the problem, by adapting the targets of the deictic gestures, the speech, and the synchronization between speech and gestures. Starting from initial learning activities of a few minutes, the system can automatically generate hours of quality on-line learning activities. An evaluation by computer graphics, computer animation, and education research experts reveals that the automatically generated learning activities have animation quality that is comparable to that of the original activities.

[1]  Johnson Wlewis,et al.  Integrating pedagogical agents into virtual environments;Presence , 1998 .

[2]  A. Graesser,et al.  Embodied conversational agents as conversational partners , 2009 .

[3]  Takaaki Kuratate,et al.  Making a thinking-talking head , 2007, AVSP.

[4]  Charles B. Callaway,et al.  Cosmo: A Life-like Animated Pedagogical Agent with Deictic Believability (cid:3) , 1997 .

[5]  Pierre Feyereisen,et al.  Further investigation on the mnemonic effect of gestures: Their meaning matters , 2006 .

[6]  Nicoletta Adamo-Villani,et al.  Charismatic and eloquent instructor avatars with scriptable gesture , 2014, SIGGRAPH '14.

[7]  James C. Lester,et al.  Achieving Affective Impact: Visual Emotive Communication in Lifelike Pedagogical Agents , 1999 .

[8]  Monica Bucciarelli,et al.  The Deep Versus the Shallow: Effects of Co-Speech Gestures in Learning From Discourse , 2008, Cogn. Sci..

[9]  R. Klatzky,et al.  Teachers’ gestures facilitate students’ learning: A lesson in symmetry , 2003 .

[10]  W. Lewis Johnson,et al.  Animated Agents for Procedural Training in Virtual Reality: Perception, Cognition, and Motor Control , 1999, Appl. Artif. Intell..

[11]  Pedagogical Content Knowledge (PCK) , 2014 .

[12]  Norbert Reithinger,et al.  VirtualHuman: dialogic and affective interaction with virtual characters , 2006, ICMI '06.

[13]  R. B. Church,et al.  The role of gesture in memory and social communication , 2007 .

[14]  R. L. Cohen,et al.  The mnemonic effect of speech gestures: Pantomimic and non-pantomimic gestures compared , 1992 .

[15]  G. Theonas,et al.  Employing Virtual Lecturers' Facial Expressions in Virtual Educational Environments , 2008, Int. J. Virtual Real..

[16]  Robert K. Atkinson,et al.  Animated pedagogical agents: does their degree of embodiment impact learning from static or animated worked examples? , 2007 .

[17]  A. Samuel,et al.  The role of speech-gesture congruency and delay in remembering action events , 2011 .

[18]  R. Mayer,et al.  Interactive Multimodal Learning Environments , 2007 .

[19]  Reed G. Williams,et al.  An Extended Visit with Dr. Fox: Validity of Student Satisfaction with Instruction Ratings after Repeated Exposures To A Lecturer , 1977 .

[20]  Jeffrey Holmes,et al.  Designing agents to support learning by explaining , 2007, Comput. Educ..

[21]  Leonard A. Annetta,et al.  Creating Presence and Community in a Synchronous Virtual Learning Environment Using Avatars , 2006 .

[22]  James C. Lester,et al.  Animated pedagogical agents and problem-solving effectiveness: a large-scale empirical evaluation , 1997 .

[23]  Allen Munro,et al.  Integrating Pedagogical Agents into Virtual Environments , 1998, Presence.

[24]  Shahrzad Mahootian,et al.  The Role of Gesture in Bilingual Education: Does Gesture Enhance Learning? , 2004 .

[25]  James C. Lester,et al.  Lifelike Pedagogical Agents for Mixed-initiative Problem Solving in Constructivist Learning Environments , 2004, User Modeling and User-Adapted Interaction.

[26]  W. Lewis Johnson,et al.  Steve: an animated pedagogical agent for procedural training in virtual environments , 1997, SGAR.

[27]  Susan Goldin-Meadow,et al.  Children Learn When Their Teacher's Gestures and Speech Differ , 2005, Psychological science.

[28]  Hussin Hanipah.  Pedagogical Content Knowledge (PCK) , 2010 .

[29]  Trent W. Lewis,et al.  PETA – a Pedagogical Embodied Teaching Agent , 2008 .

[30]  Susan Goldin-Meadow,et al.  Hands in the air: using ungrounded iconic gestures to teach children conservation of quantity. , 2008, Developmental psychology.

[31]  Nicoletta Adamo-Villani,et al.  Two Novel Technologies for Accessible Math and Science Education , 2008, IEEE MultiMedia.