Understanding Attention to Adaptive Hints in Educational Games: An Eye-Tracking Study

This paper presents a user study that investigates the factors affecting student attention to user-adaptive hints during interaction with an educational computer game. The study focuses on Prime Climb, an educational game designed to provide individualized support for learning number factorization skills in the form of textual hints based on a model of student learning. We use eye-tracking data to capture user attention patterns on the game adaptive-hints and present results on how user performance, hint timing, and attitude toward getting help all affect the degree to which students attend to hints. We also show that improved attention to hints is a worthwhile goal, at least in Prime Climb, because when they are attended to hints can positively affect a student’s performance with the game.

[1]  Bruce Edmonds,et al.  Socially Intelligent Agents: Creating Relationships With Computers And Robots , 2013 .

[2]  John L. Sibert,et al.  The reading assistant: eye gaze triggered auditory prompting for reading remediation , 2000, UIST '00.

[3]  Cristina Conati,et al.  Inferring Learning from Gaze Data during Interaction with an Environment to Support Self-Regulated Learning , 2013, AIED.

[4]  John R. Anderson,et al.  Broader Bandwidth in Student Modeling: What if ITS were "Eye"TS? , 2000, Intelligent Tutoring Systems.

[5]  Peter Brusilovsky,et al.  User-adaptive explanatory program visualization: evaluation and insights from eye movements , 2010, User Modeling and User-Adapted Interaction.

[6]  Roel Bosker,et al.  Modeled Variance in Two-Level Models , 1994 .

[7]  Lei Qu,et al.  Classifying Learner Engagement through Integration of Multiple Data Sources , 2006, AAAI.

[8]  Kasia Muldner,et al.  Investigating the Utility of Eye-Tracking Information on Affect and Reasoning for User Modeling , 2009, UMAP.

[9]  Ben Kirman,et al.  Practical, appropriate, empirically-validated guidelines for designing educational games , 2011, CHI.

[10]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[11]  Antonija Mitrovic,et al.  Fifteen years of constraint-based tutors: what we have achieved and where we are going , 2011, User Modeling and User-Adapted Interaction.

[12]  Beverly Park Woolf,et al.  Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning , 2008 .

[13]  Ryan Shaun Joazeiro de Baker,et al.  Developing a generalizable detector of when students game the system , 2008, User Modeling and User-Adapted Interaction.

[14]  Vincent Aleven,et al.  Learner Differences in Hint Processing , 2012, EDM.

[15]  John R. Anderson,et al.  Spanning seven orders of magnitude: a challenge for cognitive modeling , 2002, Cogn. Sci..

[16]  Cristina Conati,et al.  Empirically building and evaluating a probabilistic model of user affect , 2009, User Modeling and User-Adapted Interaction.

[17]  W. Lewis Johnson,et al.  Detecting the Learner's Motivational States in An Interactive Learning Environment , 2005, AIED.

[18]  David Gibson,et al.  Games And Simulations in Online Learning: Research and Development Frameworks , 2006 .

[19]  Joseph E. Beck,et al.  Engagement tracing: using response times to model student disengagement , 2005, AIED.

[20]  Vincent Aleven,et al.  The Help Tutor: Does Metacognitive Feedback Improve Students' Help-Seeking Actions, Skills and Learning? , 2006, Intelligent Tutoring Systems.

[21]  Susan Bull,et al.  Visual Attention in Open Learner Model Presentations: An Eye-Tracking Investigation , 2007, User Modeling.

[22]  David C. Lane,et al.  Modelling as learning , 1990 .

[23]  Joel A Dubin,et al.  Advantages of mixed effects models over traditional ANOVA models in developmental studies: a worked example in a mouse model of fetal alcohol syndrome. , 2007, Developmental psychobiology.

[24]  Vincent Aleven,et al.  Toward Tutoring Help Seeking: Applying Cognitive Modeling to Meta-cognitive Skills , 2004, Intelligent Tutoring Systems.

[25]  James C. Lester,et al.  Modeling Task-Based vs. Affect-based Feedback Behavior in Pedagogical Agents: An Inductive Approach , 2009, AIED.

[26]  James C. Lester,et al.  Predicting Student Self-regulation Strategies in Game-Based Learning Environments , 2012, ITS.

[27]  Cristina Conati,et al.  Using Bayesian Networks to Manage Uncertainty in Student Modeling , 2002, User Modeling and User-Adapted Interaction.

[28]  Cristina Conati,et al.  An Analysis of Attention to Student - Adaptive Hints in an Educational Game , 2012, ITS.

[29]  Antonija Mitrovic,et al.  Evaluating the effectiveness of feedback in SQL-Tutor , 2000, Proceedings International Workshop on Advanced Learning Technologies. IWALT 2000. Advanced Learning Technology: Design and Development Issues.

[30]  Richard Van Eck Building Artificially Intelligent Learning Games , 2007 .

[31]  Hua Wang,et al.  Empathic tutoring software agents using real-time eye tracking , 2006, ETRA '06.

[32]  Cristina Conati,et al.  Individual user characteristics and information visualization: connecting the dots through eye tracking , 2013, CHI.

[33]  Cristina Conati,et al.  Exploring gaze data for determining user learning with an interactive simulation , 2012, UMAP.

[34]  Joseph H. Goldberg,et al.  Comparing information graphics: a critical look at eye tracking , 2010, BELIV '10.

[35]  Ryan Shaun Joazeiro de Baker,et al.  Detecting Student Misuse of Intelligent Tutoring Systems , 2004, Intelligent Tutoring Systems.

[36]  Jonathan P. Rowe,et al.  Integrating Learning, Problem Solving, and Engagement in Narrative-Centered Learning Environments , 2011, Int. J. Artif. Intell. Educ..

[37]  M. Just,et al.  The psychology of reading and language comprehension , 1986 .

[38]  Moffat Mathews,et al.  Do Your Eyes Give It Away? Using Eye Tracking Data to Understand Students' Attitudes towards Open Student Model Representations , 2012, ITS.

[39]  Neil T. Heffernan,et al.  Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems , 2006, Intelligent Tutoring Systems.

[40]  Vincent Aleven,et al.  Using Tutors to Improve Educational Games , 2011, AIED.

[41]  Nicholas Taylor,et al.  Digital Games for Education: When Meanings Play , 2007, DiGRA Conference.

[42]  Beverly Park Woolf,et al.  A Dynamic Mixture Model to Detect Student Motivation and Proficiency , 2006, AAAI.

[43]  Cristina Conati,et al.  Evaluating Adaptive Feedback in an Educational Computer Game , 2009, IVA.

[44]  Andrew Olney,et al.  Gaze tutor: A gaze-reactive intelligent tutoring system , 2012, Int. J. Hum. Comput. Stud..

[45]  Cristina Conati,et al.  Comparing and Combining Eye Gaze and Interface Actions for Determining User Learning with an Interactive Simulation , 2013, UMAP.

[46]  Jonathan P. Rowe,et al.  Modeling User Knowledge with Dynamic Bayesian Networks in Interactive Narrative Environments , 2010, AIIDE.

[47]  Cristina Conati,et al.  Eye-tracking for user modeling in exploratory learning environments: An empirical evaluation , 2007, Knowl. Based Syst..

[48]  Vincent P. Wade,et al.  Adaptive Educational Games: Providing Non-invasive Personalised Learning Experiences , 2008, 2008 Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning.

[49]  François Albera,et al.  Intermédialités. Histoire et théorie des arts, des lettres et des techniques. n° 17, printemps 2011 , 2011 .

[50]  Antonija Mitrovic,et al.  Assessing the Impact of Positive Feedback in Constraint-Based Tutors , 2008, Intelligent Tutoring Systems.

[51]  Arthur C. Graesser,et al.  Artificial Intelligence in Education - Building Learning Systems that Care: From Knowledge Representation to Affective Modelling, Volume 200 Frontiers in Artificial Intelligence and Applications , 2009 .

[52]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[53]  Cristina Conati,et al.  Socially Intelligent Agents in Educational Games , 2002 .