Do Pedagogical Agents Enhance Software Training?

This study investigates whether a tutorial for software training can be enhanced by adding a pedagogical agent, and whether the type of agent matters i.e., cognitive, motivational, or mixed. The cognitive agent was designed to stimulate students to process their experiences actively. The motivational agent was designed to increase perceived task relevance and self-efficacy beliefs. A mixed agent combined these features. Process and product data were recorded during and after software training of students from the upper grades of vocational education M age = 16.2 years. Comparison of scores on performance measures during training revealed a significant advantage of working with the motivational and mixed agents for two important motivational mediators for learning i.e., strategy systematicity and mood. All students were highly successful during training, improving from an average 30% task completion score on the pretest to a 77% posttest score. On a retention measure 3 weeks later, task completion was still at 66%. Working with the motivational and control agents yielded significantly higher retention scores, whereas working with the motivational and mixed agents led to significantly higher scores on task relevance and self-efficacy beliefs after training. The discussion reflects on the possibilities for improving the internal and external properties of the agents.

[1]  R. Moreno,et al.  Students' choice of animated pedagogical agents in science learning: A test of the similarity-attraction hypothesis on gender and ethnicity , 2006 .

[2]  P. Pintrich,et al.  Motivation in Education: Theory, Research, and Applications , 1995 .

[3]  Hans van der Meij,et al.  What Makes Up A Procedure , 2002 .

[4]  Kasia Muldner,et al.  The Impact of Animated Pedagogical Agents on Girls' and Boys' Emotions, Attitudes, Behaviors and Learning , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[5]  Alexander Renkl,et al.  Interactive Example-Based Learning Environments: Using Interactive Elements to Encourage Effective Processing of Worked Examples , 2007 .

[6]  Scotty D. Craig,et al.  The Deep-Level-Reasoning-Question Effect: The Role of Dialogue and Deep-Level-Reasoning Questions During Vicarious Learning , 2006 .

[7]  Yanghee Kim,et al.  Pedagogical Agents as Learning Companions: The Role of Agent Competency and Type of Interaction , 2006 .

[8]  Bonnie E. John,et al.  The Strategic Use of Complex Computer Systems , 2000, Hum. Comput. Interact..

[9]  Joyce Karreman,et al.  Adding Motivational Elements to an Instruction Manual for Seniors: Effects on Usability and Motivation , 2007 .

[10]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[11]  J. Keller Motivational Design for Learning and Performance: The ARCS Model Approach , 2009 .

[12]  Wilmot Li,et al.  Design principles for visual communication , 2011, Commun. ACM.

[13]  J. Keller Development and use of the ARCS model of instructional design , 1987 .

[14]  H. Tager-Flusberg Brief report: Current theory and research on language and communication in autism , 1996, Journal of autism and developmental disorders.

[15]  R. Mayer,et al.  Engaging students in active learning: The case for personalized multimedia messages. , 2000 .

[16]  J. Keller Using the ARCS Motivational Process in Computer-Based Instruction and Distance Education , 1999 .

[17]  Heather H. Mitchell,et al.  AutoTutor: A tutor with dialogue in natural language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[18]  J. Sweller,et al.  Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions , 2005 .

[19]  John Keller,et al.  Learner motivation and E-learning design: A multinationally validated process , 2004 .

[20]  Jeffrey Holmes,et al.  Designing agents to support learning by explaining , 2007, Comput. Educ..

[21]  Susan Wiedenbeck,et al.  End-user training: an empirical study comparing on-line practice methods , 1995, CHI '95.

[22]  Hsiao-Lin Tuan,et al.  Using ARCS Model to Promote 11th Graders' Motivation and Achievement in Learning about Acids and Bases , 2005 .

[23]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[24]  Alexander Renkl,et al.  Learning from Worked-Out-Examples: A Study on Individual Differences , 1997, Cogn. Sci..

[25]  J. Eccles,et al.  Motivational beliefs, values, and goals. , 2002, Annual review of psychology.

[26]  Amy B. Adcock,et al.  Animated Agents Teaching Helping Skills in an Online Environment: A Pilot Study , 2007 .

[27]  H. VAN DER MEIJ,et al.  The four components of a procedure , 2004, IEEE Transactions on Professional Communication.

[28]  Yanghee Kim,et al.  Pedagogical Agent Design: The Impact of Agent Realism, Gender, Ethnicity, and Instructional Role , 2004, Intelligent Tutoring Systems.

[29]  Richard E. Mayer,et al.  A Personalization Effect in Multimedia Learning: Students Learn Better When Words Are in Conversational Style Rather Than Formal Style. , 2004 .

[30]  James C. Lester,et al.  The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact With Animated Pedagogical Agents? , 2001 .

[31]  S. Derry,et al.  Learning from Examples: Instructional Principles from the Worked Examples Research , 2000 .

[32]  Janet C. Read,et al.  Validating the Fun Toolkit: an instrument for measuring children’s opinions of technology , 2008, Cognition, Technology & Work.

[33]  R. Atkinson,et al.  Transitioning From Studying Examples to Solving Problems: Effects of Self-Explanation Prompts and Fading Worked-Out Steps. , 2003 .

[34]  Jason Huett,et al.  Improving the Motivation and Retention of Online Students Through the Use of ARCS-Based E-Mails , 2008 .

[35]  Susan Wiedenbeck,et al.  An activity-based analysis of hands-on practice methods , 2008, J. Comput. Assist. Learn..

[36]  Justine Cassell,et al.  Virtual peers as partners in storytelling and literacy learning , 2003, J. Comput. Assist. Learn..

[37]  Ning Wang,et al.  Experimental evaluation of polite interaction tactics for pedagogical agents , 2005, IUI.

[38]  Janet Palmer,et al.  Affective guidance of intelligent agents: How emotion controls cognition , 2009, Cognitive Systems Research.

[39]  Robert K. Atkinson,et al.  Fostering multimedia learning of science: Exploring the role of an animated agent's image , 2007, Comput. Educ..

[40]  Rinat B. Rosenberg-Kima,et al.  Interface agents as social models for female students: The effects of agent visual presence and appearance on female students' attitudes and beliefs , 2008, Comput. Hum. Behav..

[41]  Regina Vollmeyer,et al.  Motivation and metacognition when learning a complex system , 1999 .

[42]  Kate S. Hone,et al.  Empathic agents to reduce user frustration: The effects of varying agent characteristics , 2006, Interact. Comput..

[43]  Peter M. Hastings,et al.  Research Methods Tutor: Evaluation of a dialogue-based tutoring system in the classroom , 2008, Behavior research methods.

[44]  Wai-Tat Fu,et al.  Resolving the paradox of the active user: stable suboptimal performance in interactive tasks , 2004, Cogn. Sci..

[45]  A. Bandura Self-Efficacy: The Exercise of Control , 1997, Journal of Cognitive Psychotherapy.

[46]  Ard W. Lazonder,et al.  Error-information in tutorial documentation: Supporting users' errors to facilitate initial skill learning , 1995, Int. J. Hum. Comput. Stud..

[47]  G. Clarebout,et al.  Do pedagogical agents make a difference to student motivation and learning , 2011 .

[48]  Nicole Loorbach,et al.  The Effects of Motivational Elements in User Instructions , 2006 .

[49]  Stacy Marsella,et al.  Interactive pedagogical drama , 2000, AGENTS '00.

[50]  Regina Vollmeyer,et al.  Motivational Effects on Self-Regulated Learning with Different Tasks , 2006 .

[51]  Kasia Muldner,et al.  The Effect of Motivational Learning Companions on Low Achieving Students and Students with Disabilities , 2010, Intelligent Tutoring Systems.

[52]  J. Keller Motivational Design for Learning and Performance , 2010 .

[53]  Roxana Moreno,et al.  The Cambridge Handbook of Multimedia Learning: Multimedia Learning with Animated Pedagogical Agents , 2005 .

[54]  James C. Lester,et al.  Modeling and evaluating empathy in embodied companion agents , 2007, Int. J. Hum. Comput. Stud..

[55]  Magnus Haake,et al.  Design of animated pedagogical agents - A look at their look , 2006, Int. J. Hum. Comput. Stud..

[56]  F. Paas,et al.  Measurement of Cognitive Load in Instructional Research , 1994, Perceptual and motor skills.

[57]  Yanghee Kim,et al.  Pedagogical agents as learning companions: the impact of agent emotion and gender , 2007, J. Comput. Assist. Learn..

[58]  Peter Reimann,et al.  The role of self-explanation in learning to use a spreadsheet through examples , 2000, J. Comput. Assist. Learn..

[59]  Richard E. Mayer,et al.  Multimedia learning in an interactive self-explaining environment: What works in the design of agent , 2003 .

[60]  Robert K. Atkinson,et al.  Animated pedagogical agents: does their degree of embodiment impact learning from static or animated worked examples? , 2007 .

[61]  Detlev Leutner Double-fading support - a training approach to complex software systems , 2000, J. Comput. Assist. Learn..

[62]  John M. Carroll Minimalism beyond the Nurnberg funnel , 1998 .

[63]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[64]  Andrew Ortony,et al.  The Referential Structure of the Affective Lexicon , 1987, Cogn. Sci..

[65]  S. Schworm,et al.  Learning argumentation skills through the use of prompts for self-explaining examples. , 2007 .

[66]  R. Mayer,et al.  Fostering social agency in multimedia learning: Examining the impact of an animated agent’s voice ☆ , 2005 .

[67]  H. GLASBEEK,et al.  Solving problems on your own: how do exercises in tutorials interact with software learners' level of goal-orientedness? , 2004, IEEE Transactions on Professional Communication.

[68]  John Millar Carroll The Nurnberg Funnel: Designing Minimalist Instruction for Practical Computer Skill , 1990 .

[69]  Jonathan Klein,et al.  This computer responds to user frustration: Theory, design, and results , 2002, Interact. Comput..

[70]  John M. Carroll,et al.  Principles and Heuristics for Designing Minimalist Instruction , 1998 .

[71]  Erin Shaw,et al.  Animated Pedagogical Agents: An Opportunity to be Grasped? , 2002 .

[72]  Maria Bannert,et al.  The effects of training wheels and self-learning materials in software training , 2008, J. Comput. Assist. Learn..

[73]  R. Mayer,et al.  Multimedia Learning: The Promise of Multimedia Learning , 2001 .

[74]  Byron Reeves,et al.  The effects of animated characters on anxiety, task performance, and evaluations of user interfaces , 2000, CHI.

[75]  Susanne van Mulken,et al.  The impact of animated interface agents: a review of empirical research , 2000, Int. J. Hum. Comput. Stud..

[76]  Ning Wang,et al.  The politeness effect: Pedagogical agents and learning outcomes , 2008, Int. J. Hum. Comput. Stud..

[77]  Elisabeth André,et al.  The Persona Effect: How Substantial Is It? , 1998, BCS HCI.

[78]  Steffi Domagk,et al.  Do Pedagogical Agents Facilitate Learner Motivation and Learning Outcomes? , 2010, J. Media Psychol. Theor. Methods Appl..

[79]  Donald A. Norman,et al.  How might people interact with agents , 1994, CACM.

[80]  R. Mayer,et al.  Personalized messages that promote science learning in virtual environments , 2004 .

[81]  Isabel Machado,et al.  Lifelong training with Vincent, a web-based pedagogical agent , 2002 .

[82]  Mary Hegarty,et al.  When static media promote active learning: annotated illustrations versus narrated animations in multimedia instruction. , 2005, Journal of experimental psychology. Applied.

[83]  Hans van der Meij,et al.  Designing for user cognition and affect in software instructions , 2008 .

[84]  John M. Keller,et al.  Effectiveness of motivationally adaptive computer-assisted instruction on the dynamic aspects of motivation , 2001 .

[85]  Maria Virvou,et al.  Evaluating the persona effect of an interface agent in a tutoring system , 2002, J. Comput. Assist. Learn..