How Effective are Pedagogical Agents for Learning? A Meta-Analytic Review

Research on the use of software programs and tools such as pedagogical agents has peaked over the last decade. Pedagogical agents are on-screen characters that facilitate instruction. This meta-analysis examined the effect of using pedagogical agents on learning by reviewing 43 studies involving 3,088 participants. Analysis of the results indicated that pedagogical agents produced a small but significant effect on learning. The overall mean effect size was moderated by the contextual and methodological features of the studies. The findings revealed that the use of pedagogical agents were more beneficial for K-12 students than post-secondary students. Pedagogical agents that communicated with students using on-screen text facilitated learning more effectively than agents that communicated using narration. The findings of this study have implications for advancing theory and practice, as well as highlighting productive future directions for research.

[1]  Maria Virvou,et al.  Evaluating the Impact of Interface Agents in an Intelligent Tutoring Systems Authoring Tool , 2001 .

[2]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[3]  Hsin I Yung.  Effects of an Animated Agent with Instructional Strategies in Facilitating Student Achievement of Different Educational Objectives in Multimedia Learning. , 2009 .

[4]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[5]  Scotty D. Craig,et al.  Animated Pedagogical Agents in Multimedia Educational Environments: Effects of Agent Properties, Picture Features, and Redundancy , 2002 .

[6]  Petek Askar,et al.  The effect of an embedded pedagogical agent on the students' science achievement , 2008, Interact. Technol. Smart Educ..

[7]  J. Burgoon,et al.  Interactivity in human–computer interaction: a study of credibility, understanding, and influence , 2000 .

[8]  George Veletsianos,et al.  Contextually relevant pedagogical agents: Visual appearance, stereotypes, and first impressions and their impact on learning , 2010, Comput. Educ..

[9]  M. Malbrán.  The Cambridge Handbook of Multimedia Learning , 2007 .

[10]  Paul Ginns.  Meta-Analysis of the Modality Effect. , 2005 .

[11]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[12]  Richard E. Mayer,et al.  The Cambridge Handbook of Multimedia Learning: Cognitive Theory of Multimedia Learning , 2005 .

[13]  Arthur C. Graesser,et al.  The Impact of Conversational Navigational Guides on the Learning, Use, and Perceptions of Users of a Web Site , 2003, AMKM.

[14]  J. Sweller.  Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load , 2010 .

[15]  Apa Publications,et al.  Reporting standards for research in psychology: why do we need them? What might they be? , 2008, The American psychologist.

[16]  Maria Virvou,et al.  Evaluating the persona effect of an interface agent in a tutoring system , 2002, J. Comput. Assist. Learn..

[17]  A. Graesser,et al.  Embodied conversational agents as conversational partners , 2009 .

[18]  Soyoung Kim,et al.  Designing nonverbal communication for pedagogical agents: When less is more , 2009, Comput. Hum. Behav..

[19]  Richard E. Mayer,et al.  Cognitive Theory of Multimedia Learning , 2021, The Cambridge Handbook of Multimedia Learning.

[20]  Sanghoon Park,et al.  The effects of seductive augmentation and agent role on learning interest, achievement, and attitude , 2005 .

[21]  Martin Reisslein,et al.  Using Virtual Peers to Guide Visual Attention During Learning , 2010, J. Media Psychol. Theor. Methods Appl..

[22]  Aaron Doering,et al.  Enali: A Research and Design Framework for Virtual Characters and Pedagogical Agents , 2009 .

[23]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[24]  Rinat B. Rosenberg-Kima,et al.  Changing middle-school students' attitudes and performance regarding engineering with computer-based social models , 2009, Comput. Educ..

[25]  Jeffrey Holmes,et al.  Designing agents to support learning by explaining , 2007, Comput. Educ..

[26]  Elisabeth André,et al.  The Persona Effect: How Substantial Is It? , 1998, BCS HCI.

[27]  Steffi Domagk,et al.  Do Pedagogical Agents Facilitate Learner Motivation and Learning Outcomes? , 2010, J. Media Psychol. Theor. Methods Appl..

[28]  Richard Van Eck,et al.  Etiquette and Efficacy in Animated Pedagogical Agents: The Role of Stereotypes , 2002 .

[29]  Jeroen J. G. van Merriënboer,et al.  The Cambridge Handbook of Multimedia Learning: The Four-Component Instructional Design Model: Multimedia Principles in Environments for Complex Learning , 2005 .

[30]  Richard Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[31]  Charles Miller,et al.  Conversing with pedagogical agents: A phenomenological exploration of interacting with digital entities , 2008, Br. J. Educ. Technol..

[32]  Richard E. Mayer,et al.  Multimedia learning in an interactive self-explaining environment: What works in the design of agent , 2003 .

[33]  Olusola O. Adesope,et al.  Verbal redundancy in multimedia learning environments: A meta-analysis. , 2012 .

[34]  R. Clark,et al.  The Questionable Benefits of Pedagogical Agents: Response to Veletsianos , 2007 .

[35]  J. Sweller.  Implications of Cognitive Load Theory for Multimedia Learning , 2005, The Cambridge Handbook of Multimedia Learning.

[36]  Richard E. Clark,et al.  Cognitive and Affective Benefits of an Animated Pedagogical Agent for Learning English as a Second Language , 2006 .

[37]  A. L. Baylor.  Expanding preservice teachers' metacognitive awareness of instructional planning through pedagogical agents , 2002 .

[38]  R. Mayer.  The Cambridge Handbook of Multimedia Learning: Principles of Multimedia Learning Based on Social Cues : Personalization, Voice, and Image Principles , 2005 .

[39]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[40]  Zsuzsanna I. Abrams,et al.  Learning with laura: investigating the effects of a pedagogical agent on spanish lexical acquisition , 2009 .

[41]  Kevin Kirk.  Performance, perception, and choice of animated pedagogical agent , 2008 .

[42]  F. Paas,et al.  Variability of Worked Examples and Transfer of Geometrical Problem-Solving Skills: A Cognitive-Load Approach , 1994 .

[43]  George Veletsianos,et al.  How do learners respond to pedagogical agents that deliver social-oriented non-task messages? Impact on student learning, perceptions, and experiences , 2012, Comput. Hum. Behav..

[44]  Robin Kay,et al.  Analysing the Effectiveness of Learning Objects for Secondary School Science Classrooms , 2009 .

[45]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .

[46]  Arthur C. Graesser,et al.  AutoTutor: an intelligent tutoring system with mixed-initiative dialogue , 2005, IEEE Transactions on Education.

[47]  Alex J Sutton,et al.  Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. , 2002, International journal of epidemiology.

[48]  James C. Lester,et al.  Achieving Affective Impact: Visual Emotive Communication in Lifelike Pedagogical Agents , 1999 .

[49]  Amy B. Adcock,et al.  Teaching Effective Helping Skills at a Distance: The Development of Project CATHIE. , 2006 .

[50]  Kathryn Kennedy,et al.  Offering Preservice Teachers Field Experiences in K-12 Online Learning , 2012 .

[51]  Byron Reeves,et al.  The effects of animated characters on anxiety, task performance, and evaluations of user interfaces , 2000, CHI.

[52]  R. Mayer,et al.  Fostering social agency in multimedia learning: Examining the impact of an animated agent’s voice ☆ , 2005 .

[53]  Punya Mishra,et al.  RUNNING HEAD: PEDAGOGICAL SOFTWARE AGENTS All or Nothing: Levels of Sociability of a Pedagogical Software Agent and its Impact on Student Perceptions and Learning , 2005 .

[54]  B. Tabachnick,et al.  Using multivariate statistics, 5th ed. , 2007 .

[55]  Robert K. Atkinson,et al.  Fostering multimedia learning of science: Exploring the role of an animated agent's image , 2007, Comput. Educ..

[56]  Susanne van Mulken,et al.  The impact of animated interface agents: a review of empirical research , 2000, Int. J. Hum. Comput. Stud..

[57]  James C. Lester,et al.  The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact With Animated Pedagogical Agents? , 2001 .

[58]  Melissa Murray,et al.  Computerized Pedagogical Agents as an Educational Means for Developing Physical Self-Efficacy and Encouraging Activity in Youth , 2010 .

[59]  G. Clarebout,et al.  Do pedagogical agents make a difference to student motivation and learning , 2011 .

[60]  Lih-Shyang Chen,et al.  Building a General Purpose Pedagogical Agent in a Web-Based Multimedia Clinical Simulation System for Medical Education , 2009, IEEE Transactions on Learning Technologies.

[61]  R. Atkinson.  Optimizing learning from examples using animated pedagogical agents. , 2002 .

[62]  S. Yusuf,et al.  Strengths and limitations of meta-analysis: larger studies may be more reliable. , 1997, Controlled clinical trials.

[63]  J. Sweller,et al.  The Cambridge Handbook of Multimedia Learning: The Modality Principle in Multimedia Learning , 2005 .

[64]  Yanghee Kim,et al.  Pedagogical agents as learning companions: the impact of agent emotion and gender , 2007, J. Comput. Assist. Learn..

[65]  Agneta Gulz,et al.  Benefits of Virtual Characters in Computer Based Learning Environments: Claims and Evidence , 2004, Int. J. Artif. Intell. Educ..

[66]  L. Hedges,et al.  RESEARCH SYNTHESIS AS A SCIENTIFIC PROCESS , 2009, The Handbook of Research Synthesis and Meta-Analysis.

[67]  Richard E. Clark,et al.  Five Design Principles for Experiments on the Effects of Animated Pedagogical Agents , 2005 .

[68]  Richard E. Mayer,et al.  Principles for Managing Essential Processing in Multimedia Learning: Segmenting, Pre-training, and Modality Principles , 2005 .

[69]  Richard Van Eck,et al.  Reliability and factor structure of the Attitude Toward Tutoring Agent Scale (ATTAS) , 2005 .

[70]  John Sweller,et al.  The Cambridge Handbook of Multimedia Learning: The Split-Attention Principle in Multimedia Learning , 2005 .

[71]  John Sweller,et al.  The Split-Attention Principle in Multimedia Learning , 2021, The Cambridge Handbook of Multimedia Learning.

[72]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[73]  Roxana Moreno,et al.  The Cambridge Handbook of Multimedia Learning: Multimedia Learning with Animated Pedagogical Agents , 2005 .

[74]  John Sweller,et al.  The Modality Principle in Multimedia Learning , 2014, The Cambridge Handbook of Multimedia Learning.

[75]  Autumn B. Hostetter,et al.  When do gestures communicate? A meta-analysis. , 2011, Psychological bulletin.

[76]  Minjeong Kim,et al.  Meta-Analysis of the Effectiveness of Pedagogical Agent , 2003 .

[77]  Roxana Moreno,et al.  The Roles of Animated Pedagogical Agents' Presence and Nonverbal Communication in Multimedia Learning Environments , 2010, J. Media Psychol. Theor. Methods Appl..

[78]  Heather H. Mitchell,et al.  Social Cues in Animated Conversational Agents , 2005 .

[79]  Robert K. Atkinson,et al.  Animated pedagogical agents: does their degree of embodiment impact learning from static or animated worked examples? , 2007 .

[80]  Patricia D. Mautone,et al.  Social cues in multimedia learning: Role of speaker's voice. , 2003 .