Effects of text difficulty and readers on predicting reading comprehension from eye movements

The task of predicting reader state from readers' eye gaze is not trivial. Whilst eye movements have long been shown to reflect the reading process, the task of predicting quantified measures of reading comprehension has been attempted with unsatisfactory results. We conducted an experiment to collect eye gaze data from participants as they read texts with differing degrees of difficulty. Participants were sourced as being either first or second English language readers. We investigated the effects that reader background and text difficulty have predicting reading comprehension. The results indicate that prediction rates are similar for first and second language readers. The best combination is where the concept level is one level higher than the readability level. The optimal predictors are ELM+NN and Random Forests as they consistently produced the lowest MSEs on average. These findings are a promising step forward to predicting reading comprehension. The intention is to use such predictions in adaptive eLearning environments.

[1]  Chih-Ming Chen,et al.  Intelligent web-based learning system with personalized learning path guidance , 2008, Comput. Educ..

[2]  Herman Dwi Surjono.  The Evaluation of a Moodle Based Adaptive e-Learning System , 2014 .

[3]  Tamás D. Gedeon,et al.  Predicting reading comprehension scores from eye movements using artificial neural networks and fuzzy output error , 2014, Artif. Intell. Res..

[4]  John L. Sibert,et al.  The reading assistant: eye gaze triggered auditory prompting for reading remediation , 2000, UIST '00.

[5]  S. Martinez-Conde.  Fixational eye movements in normal and pathological vision. , 2006, Progress in brain research.

[6]  Geoffrey Underwood,et al.  Eye Fixations Predict Reading Comprehension: The Relationships between Reading Skill, Reading Speed, and Visual Inspection , 1990, Language and speech.

[7]  Christian Gütl,et al.  AdeLE (Adaptive e-Learning with Eye-Tracking): Theoretical Background, System Architecture and Application Scenarios , 2005 .

[8]  F. Kareal,et al.  Adaptivity in e-learning , 2006 .

[9]  S. Liversedge,et al.  Saccadic eye movements and cognition , 2000, Trends in Cognitive Sciences.

[10]  Alexandros Paramythis,et al.  Adaptive Learning Environments and e-Learning Standards. , 2004 .

[11]  K Rayner,et al.  Reading without a fovea. , 1979, Science.

[12]  Haijun Kang.  Understanding online reading through the eyes of first and second language readers: An exploratory study , 2014, Comput. Educ..

[13]  Paul P. Maglio,et al.  A robust algorithm for reading detection , 2001, PUI '01.

[14]  Marco Porta,et al.  e5Learning, an E-Learning Environment Based on Eye Tracking , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[15]  D. Romer,et al.  Do Students Go to Class? Should They? , 1993 .

[16]  Mike Rinck,et al.  Chapter 16 – Eye Movement Measures to Study Global Text Processing , 2003 .

[17]  Andreas Dengel,et al.  Eye movements as implicit relevance feedback , 2008, CHI Extended Abstracts.

[18]  K. Rayner,et al.  Eye Movements as Reflections of Comprehension Processes in Reading , 2006 .

[19]  K. Rayner,et al.  Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences , 1982, Cognitive Psychology.

[20]  Brian P. Bailey,et al.  Using Eye Gaze Patterns to Identify User Tasks , 2004 .

[21]  P. Baranyi,et al.  Definition and synergies of cognitive infocommunications , 2012 .

[22]  Herman Dwi Surjono.  The Design of Adaptive E-Learning System based on Student ’ s Learning Styles , 2011 .

[23]  Päivi Majaranta,et al.  Design issues of iDICT: a gaze-assisted translation aid , 2000, ETRA.

[24]  Ziming Liu,et al.  Reading behavior in the digital environment: Changes in reading behavior over the past ten years , 2005, J. Documentation.

[25]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[26]  Alexiei Dingli,et al.  Adaptive eBook , 2014, 2014 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL2014).

[27]  Andrew Olney,et al.  Gaze tutor: A gaze-reactive intelligent tutoring system , 2012, Int. J. Hum. Comput. Stud..

[28]  Tamás D. Gedeon,et al.  Fuzzy Output Error as the Performance Function for Training Artificial Neural Networks to Predict Reading Comprehension from Eye Gaze , 2014, ICONIP.

[29]  Ruth Woodfield,et al.  Gender differences in undergraduate attendance rates , 2006 .

[30]  Pascual Martínez-Gómez,et al.  Recognition of understanding level and language skill using measurements of reading behavior , 2014, IUI.

[31]  Aidan Kehoe,et al.  Using eye tracking technology to identify visual and verbal learners , 2011, 2011 IEEE International Conference on Multimedia and Expo.