Feedback source modality effects on training outcomes in a serious game: Pedagogical agents make a difference

In-game real-time feedback produced significantly higher performance outcomes.Feedback delivered by pedagogical agents resulted in largest retention outcomes.Feedback delivered as audio alone produced negative gains in assessments outcomes.Agent interface modalities did not vary across performance and mental demand scores.Feedback delivered as audio alone resulted in significantly lower mental demand. The aim of this research is to enhance game-based training applications to support educational events in the absence of live instruction. The overarching purpose of the presented study was to explore available tools for integrating intelligent tutoring communications in game-based learning platforms and to examine theory-based techniques for delivering explicit feedback in such environments. The primary tool influencing the design of this research was the open-source Generalized Intelligent Framework for Tutoring (GIFT), a modular domain-independent architecture that provides the tools and methods to author, deliver, and evaluate intelligent tutoring technologies within any instructional domain. Influenced by research surrounding social cognitive theory and cognitive load theory, the resulting experiment tested varying approaches for utilizing an Embodied Pedagogical Agent (EPA) to function as a tutor during interaction in a game-based training environment. Conditions were authored to assess the tradeoffs between embedding an EPA directly in a game, embedding an EPA in GIFT's browser-based Tutor-User Interface (TUI), or using audio prompts alone with no social grounding. The resulting data supported the application of using an EPA embedded in GIFT's TUI to provide explicit feedback during a game-based learning event. Analyses revealed conditions with an EPA situated in the TUI to be as effective as embedding the agent directly in the game environment.

[1]  J. Bailenson,et al.  Self-Representations in Immersive Virtual Environments 1 , 2008 .

[2]  James C. Lester,et al.  The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact With Animated Pedagogical Agents? , 2001 .

[3]  C. Guetl,et al.  Intelligent Pedagogical Agents in immersive virtual learning environments: A review , 2010, The 33rd International Convention MIPRO.

[4]  Scotty D. Craig,et al.  Animated Pedagogical Agents in Multimedia Educational Environments: Effects of Agent Properties, Picture Features, and Redundancy , 2002 .

[5]  L. Vygotsky.  Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[6]  Kristina Höök,et al.  Evaluating Users' Experience of a Character-enhanced Information Space , 2000, AI Commun..

[7]  Richard A. Schmidt,et al.  Frequent Augmented Feedback Can Degrade Learning: Evidence and Interpretations , 1991 .

[8]  K. Allmendinger.  Social Presence in Synchronous Virtual Learning Situations: The Role of Nonverbal Signals Displayed by Avatars , 2010 .

[9]  Amy L. Baylor,et al.  The design of motivational agents and avatars , 2011 .

[10]  Tze Wei Liew,et al.  The Effects of Peer-Like and Expert-Like Pedagogical Agents on Learners' Agent Perceptions, Task-Related Attitudes, and Learning Achievement , 2013, J. Educ. Technol. Soc..

[11]  Arthur C. Graesser,et al.  Affective Computing and Intelligent Interaction: Fourth International Conference, ACII 2011, Memphis,TN, USA, October 9-12, 2011; Proceedings, Part II ... Vision, Pattern Recognition, and Graphics) , 2011 .

[12]  Arthur C. Graesser,et al.  Guru: A Computer Tutor That Models Expert Human Tutors , 2012, ITS.

[13]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[14]  John M. Reising,et al.  Development and Evaluation of a Background Attitude Indicator , 1999 .

[15]  Frank Puppe,et al.  The appearance effect: Influences of virtual agent features on performance and motivation , 2015, Comput. Hum. Behav..

[16]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[17]  Mattijs Ghijsen,et al.  Generating Socially Appropriate Tutorial Dialog , 2004, ADS.

[18]  Robert A. Sottilare,et al.  The Generalized Intelligent Framework for Tutoring (GIFT) , 2012 .

[19]  J. Sweller,et al.  Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions , 2005 .

[20]  R Darin Ellis,et al.  NASA TLX: Software for assessing subjective mental workload , 2009, Behavior research methods.

[21]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[22]  Magnus Haake,et al.  Design of animated pedagogical agents - A look at their look , 2006, Int. J. Hum. Comput. Stud..

[23]  R. Mayer,et al.  Aids to computer-based multimedia learning , 2002 .

[24]  J. Michael Spector,et al.  Handbook of Research on Educational Communications and Technology, 3rd Edition , 2012 .

[25]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[26]  Sharon L. Oviatt,et al.  Human-centered design meets cognitive load theory: designing interfaces that help people think , 2006, MM '06.

[27]  Yanghee Kim,et al.  Simulating Instructional Roles through Pedagogical Agents , 2005, Int. J. Artif. Intell. Educ..

[28]  Jeremy N. Bailenson,et al.  A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces , 2007, CHI.

[29]  Arthur C. Graesser,et al.  Intelligent Tutoring Systems with Conversational Dialogue , 2001, AI Mag..

[30]  Jennifer J. Vogel-Walcutt,et al.  Applying the modality principle to real-time feedback and the acquisition of higher-order cognitive skills , 2012 .

[31]  James C. Lester,et al.  Life-Like Pedagogical Agents in Constructivist Multimedia Environments: Cognitive Consequences of their Interaction , 2000 .

[32]  Agneta Gulz,et al.  Benefits of Virtual Characters in Computer Based Learning Environments: Claims and Evidence , 2004, Int. J. Artif. Intell. Educ..

[33]  S. Narciss.  Feedback Strategies for Interactive Learning Tasks , 2007 .

[34]  George Veletsianos,et al.  Contextually relevant pedagogical agents: Visual appearance, stereotypes, and first impressions and their impact on learning , 2010, Comput. Educ..

[35]  Maria Virvou,et al.  Evaluating the persona effect of an interface agent in a tutoring system , 2002, J. Comput. Assist. Learn..

[36]  Robert K. Atkinson,et al.  Animated agents and learning: Does the type of verbal feedback they provide matter? , 2013, Comput. Educ..

[37]  P. Chandler,et al.  Why Some Material Is Difficult to Learn , 1994 .

[38]  Eduardo Salas,et al.  Performance Measurement in Simulation-Based Training , 2009 .

[39]  Ard W. Lazonder,et al.  Modelling human emotions for tactical decision-making games , 2013, Br. J. Educ. Technol..

[40]  James C. Lester,et al.  The persona effect: affective impact of animated pedagogical agents , 1997, CHI.

[41]  R. Mayer,et al.  A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory , 1998 .

[42]  G. Clarebout,et al.  Do pedagogical agents make a difference to student motivation and learning , 2011 .

[43]  James C. Lester.  Affect, Learning, and Delight , 2011, ACII.

[44]  Soyoung Kim,et al.  Designing nonverbal communication for pedagogical agents: When less is more , 2009, Comput. Hum. Behav..

[45]  Elisabeth André,et al.  The Persona Effect: How Substantial Is It? , 1998, BCS HCI.

[46]  J. Sweller,et al.  The Cambridge Handbook of Multimedia Learning: The Modality Principle in Multimedia Learning , 2005 .

[47]  Axel Cleeremans.  Conscious and Unconscious Processes in Cognition , 2001 .

[48]  R. Atkinson.  Optimizing learning from examples using animated pedagogical agents. , 2002 .

[49]  R. Schmidt,et al.  New Conceptualizations of Practice: Common Principles in Three Paradigms Suggest New Concepts for Training , 1992 .

[50]  Thomas Rist,et al.  Presenting through performing: on the use of multiple lifelike characters in knowledge-based presentation systems , 2001, Knowl. Based Syst..

[51]  E. Bozhovich.  Zone of Proximal Development , 2009 .

[52]  Susana Rubio,et al.  Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA‐TLX, and Workload Profile Methods , 2004 .

[53]  V. Shute.  Focus on Formative Feedback , 2007 .