Eye Tracking and Studying Examples: How Novices and Advanced Learners Study SQL Examples

Eye tracking provides information about a user’s eye gaze movements. For many years, eye tracking has been used in Human Computer Interaction (HCI) research. Similarly, research on computerised educational systems also relies heavily on students’ interactions with systems, and therefore eye tracking has been used to study and improve learning. We have recently conducted several studies on using worked examples in addition to scaffolded problem solving. The goal of the project reported in this paper was to investigate how novices and advanced students learn from examples. The study was performed in the context of SQL-Tutor, a mature Intelligent Tutoring System (ITS) that teaches SQL. We propose a new technique to analyse eye-gaze patterns named EGPA. In order to comprehend an SQL example, students require the information about tables’ names and their attributes which are available in a database schema. Thus, if students paid attention to the database schema, they would understand SQL examples more easily. We analysed students’ eye movement data from different perspectives, and found that advanced students paid more attention to database schema than novices. In future work, we will use the findings from this study to provide proactive feedback or individualised amounts of information.

[1]  Huei-Tse Hou,et al.  Visual attention for solving multiple-choice science problem: An eye-tracking analysis , 2012, Comput. Educ..

[2]  Antonija Mitrovic,et al.  Feedback Micro-engineering in EER-Tutor , 2005, AIED.

[3]  Moffat Mathews,et al.  Do Your Eyes Give It Away? Using Eye Tracking Data to Understand Students' Attitudes towards Open Student Model Representations , 2012, ITS.

[4]  Cristina Conati,et al.  A Framework for Capturing Distinguishing User Interaction Behaviors in Novel Interfaces , 2011, EDM.

[5]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[6]  Hua Wang,et al.  Empathic tutoring software agents using real-time eye tracking , 2006, ETRA '06.

[7]  Cristina Conati,et al.  Exploring gaze data for determining user learning with an interactive simulation , 2012, UMAP.

[8]  John Paulin Hansen,et al.  Evaluation of a low-cost open-source gaze tracker , 2010, ETRA.

[9]  Peter Brusilovsky,et al.  From WebEx to NavEx: Interactive Access to Annotated Program Examples , 2008, Proceedings of the IEEE.

[10]  Roman Bednarik,et al.  Potentials of Eye-Movement Tracking in Adaptive Systems , 2009 .

[11]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[12]  Kasia Muldner,et al.  Exploring Eye Tracking to Increase Bandwidth in User Modeling , 2005, User Modeling.

[13]  K. Rayner.  Eye Movements and Cognitive Processes in Reading, Visual Search, and Scene Perception , 1995 .

[14]  Beverly Park Woolf,et al.  Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning , 2008 .

[15]  Cristina Conati,et al.  Combining Unsupervised and Supervised Classification to Build User Models for Exploratory , 2009, EDM 2009.

[16]  Antonija Mitrovic,et al.  Investigating student interactions with tutorial dialogues in EER-Tutor , 2015, Research and Practice in Technology Enhanced Learning.

[17]  Antonija Mitrovic,et al.  Do novices and advanced students benefit differently from worked examples and ITS , 2013 .

[18]  Kurt VanLehn,et al.  Andes: A Coached Problem Solving Environment for Physics , 2000, Intelligent Tutoring Systems.

[19]  Vincent Aleven,et al.  The worked-example effect: Not an artefact of lousy control conditions , 2009, Comput. Hum. Behav..

[20]  Cristina Conati,et al.  Comparing and Combining Eye Gaze and Interface Actions for Determining User Learning with an Interactive Simulation , 2013, UMAP.

[21]  T. Crawford,et al.  Viewing another person's eye movements improves identification of pulmonary nodules in chest x-ray inspection. , 2010, Journal of experimental psychology. Applied.

[22]  Cristina Conati,et al.  Seeing how you're Looking - Using Real-Time Eye Gaze Data for User-Adaptive Visualization , 2013, UMAP Workshops.

[23]  Michelene T. H. Chi,et al.  Self-Explanations: How Students Study and Use Examples in Learning To Solve Problems. Technical Report No. 9. , 1987 .

[24]  Vincent Aleven,et al.  Metacognitive support promotes an effective use of instructional resources in intelligent tutoring , 2013 .

[25]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..

[26]  John R. Anderson,et al.  Broader Bandwidth in Student Modeling: What if ITS were "Eye"TS? , 2000, Intelligent Tutoring Systems.

[27]  Bruce M. McLaren,et al.  When Is It Best to Learn with All Worked Examples? , 2011, AIED.

[28]  Richard E. Mayer,et al.  Multimedia Learning. Second Edition. , 2009 .

[29]  Cristina Conati,et al.  Eye-tracking for user modeling in exploratory learning environments: An empirical evaluation , 2007, Knowl. Based Syst..

[30]  John L. Sibert,et al.  The reading assistant: eye gaze triggered auditory prompting for reading remediation , 2000, UIST '00.

[31]  Peter J. Rousseeuw,et al.  Clustering by means of medoids , 1987 .

[32]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[33]  Antonija Mitrovic,et al.  An Intelligent SQL Tutor on the Web , 2003, Int. J. Artif. Intell. Educ..

[34]  Vincent Aleven,et al.  The expertise reversal effect and worked examples in tutored problem solving , 2010 .

[35]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[36]  Susan Bull,et al.  Visual Attention in Open Learner Model Presentations: An Eye-Tracking Investigation , 2007, User Modeling.

[37]  T. Gog,et al.  Example-Based Learning: Integrating Cognitive and Social-Cognitive Research Perspectives , 2010 .

[38]  John Sweller,et al.  The worked-example effect using ill-defined problems: Learning to recognise designers' styles , 2009 .

[39]  John R. Anderson,et al.  Illustrating Principled Design: The Early Evolution of a Cognitive Tutor for Algebra Symbolization , 1998, Interact. Learn. Environ..

[40]  Agnieszka Bojko,et al.  Informative or Misleading? Heatmaps Deconstructed , 2009, HCI.

[41]  John R. Anderson,et al.  Effective Use of Intelligent Software in High School Math Classrooms , 1993 .

[42]  Antonija Mitrovic,et al.  Utilizing Eye Tracking to Improve Learning from Examples , 2014, HCI.

[43]  Neil T. Heffernan,et al.  Tutored Problem Solving vs. “Pure” Worked Examples , 2009 .

[44]  Antonija Mitrovic,et al.  Examples and Tutored Problems: How Can Self-Explanation Make a Difference to Learning? , 2013, AIED.

[45]  David G. Stork,et al.  Pattern Classification , 1973 .

[46]  Cristina Conati,et al.  Unsupervised and supervised machine learning in user modeling for intelligent learning environments , 2007, IUI '07.

[47]  Alfred Bork,et al.  Multimedia in Learning , 2001 .

[48]  T. Gog,et al.  Learning to see: Guiding students' attention via a Model's eye movements fosters learning , 2013 .

[49]  J. Sweller.  The worked example effect and human cognition , 2006 .

[50]  Katharina Scheiter,et al.  Attention guidance during example study via the model's eye movements , 2009, Comput. Hum. Behav..

[51]  Joseph H. Goldberg,et al.  Comparing information graphics: a critical look at eye tracking , 2010, BELIV '10.

[52]  Antonija Mitrovic,et al.  Experiences in Implementing Constraint-Based Modeling in SQL-Tutor , 1998, Intelligent Tutoring Systems.

[53]  Kasia Muldner,et al.  ON IMPROVING THE EFFECTIVENESS OF OPEN LEARNING ENVIRONMENTS THROUGH TAILORED SUPPORT FOR EXPLORATION , 2001 .

[54]  K. Koedinger,et al.  Exploring the Assistance Dilemma in Experiments with Cognitive Tutors , 2007 .

[55]  Peter Brusilovsky,et al.  User-adaptive explanatory program visualization: evaluation and insights from eye movements , 2010, User Modeling and User-Adapted Interaction.