A review of eye-tracking applications as tools for training

Substantial literature exists regarding how eye-tracking systems can be used to measure cognitive load and how these measurements can be useful for adapting training in real time. Much of the published literature discusses the applications and limitations of the research and typically provides recommendations for improvement. This review assesses these articles collectively to provide a clearer solution for implementing eye-tracking systems into a training environment. Although limitations exist for using eye tracking as an interface tool, gaze and pupillary response have been successfully used to reflect changes in cognitive load and are starting to be incorporated into adaptive training systems, although issues are still present with differentiating pupil responses from simultaneous psychological effects. Additionally, current eye-tracking systems and data analysis software have proven accurate enough for general use, but issues including system cost and software integration prevent this technology from becoming commercialized for use in common instructional settings.

[1]  Tao Lin,et al.  Evaluating usability based on multimodal information: an empirical study , 2006, ICMI '06.

[2]  Stephan Raidt,et al.  Gaze and face-to-face communication between a human speaker and an animated conversational agent - Mutual attention and multimodal deixis , 2008 .

[3]  Markku Tukiainen,et al.  An eye-tracking methodology for characterizing program comprehension processes , 2006, ETRA.

[4]  Gerhard Tröster,et al.  EyeMote - Towards Context-Aware Gaming Using Eye Movements Recorded from Wearable Electrooculography , 2008, Fun and Games.

[5]  F. Paas,et al.  A motivational perspective on the relation between mental effort and performance: Optimizing learner involvement in instruction , 2005 .

[6]  Fred Paas,et al.  Studying Eye Movements in Multimedia Learning , 2008 .

[7]  Katharina Scheiter,et al.  Can differences in learning strategies explain the benefits of learning from static and dynamic visualizations? , 2011, Comput. Educ..

[8]  Debra Steele-Johnson,et al.  Advanced technologies in training: Intelligent tutoring systems and virtual reality. , 1997 .

[9]  Hannele Buffy Marie Nicholson,et al.  Disfluency in Dialogue: Attention, Structure and Function , 2007 .

[10]  A. Wingfield,et al.  Pupillometry as a measure of cognitive effort in younger and older adults. , 2010, Psychophysiology.

[11]  Erol Özçelik,et al.  Why does signaling enhance multimedia learning? Evidence from eye movements , 2010, Comput. Hum. Behav..

[12]  Christian Gütl,et al.  AdeLE (Adaptive e-Learning with Eye-Tracking): Theoretical Background, System Architecture and Application Scenarios , 2005 .

[13]  Geoffrey Underwood,et al.  Restricting the field of view to investigate the perceptual spans of pianists , 2003 .

[14]  J. Wann,et al.  The effect of learning condition on perceptual anticipation, awareness, and visual search. , 2005, Human movement science.

[15]  Bahar Baran,et al.  Tracing Users' Behaviors in a Multimodal Instructional Material: An Eye-Tracking Study , 2007, HCI.

[16]  Martha E. Crosby,et al.  Visual Search and Background Complexity: Does the Forest Hide the Trees? , 2001, User Modeling.

[17]  Kate Ponton,et al.  Concepts of Interface Usability and the Enhancement of Design through Eye Tracking and Psychophysiology , 2008 .

[18]  A. Kramer,et al.  Physiological metrics of mental workload: A review of recent progress , 1990, Multiple-task performance.

[19]  Mitsuru Ishizuka,et al.  AutoSelect: What You Want Is What You Get: Real-Time Processing of Visual Attention and Affect , 2006, PIT.

[20]  Dan E. Tamir,et al.  An Effort-Based Approach to Measuring Software Usability , 2008 .

[21]  David Soto,et al.  The modulation of perceptual selection by working memory is dependent on the focus of spatial attention , 2010, Vision Research.

[22]  David N. Rapp,et al.  The value of attention aware systems in educational settings , 2006, Comput. Hum. Behav..

[23]  Edward R. Sykes Preliminary Findings of Visualization of the Interruptible Moment , 2009, HPCS.

[24]  Marco Porta,et al.  Eye-S: a full-screen input modality for pure eye-based communication , 2008, ETRA.

[25]  F. Paas,et al.  Measurement of Cognitive Load in Instructional Research , 1994, Perceptual and motor skills.

[26]  Tad T. Brunyé,et al.  When goals constrain: Eye movements and memory for goal‐Oriented map study , 2009 .

[27]  Michael E. Holmes,et al.  Clutter or content?: how on-screen enhancements affect how TV viewers scan and what they learn , 2006, ETRA.

[28]  Richard E. Mayer,et al.  How Does Text Affect the Processing of Diagrams in Multimedia Learning? , 2010, Diagrams.

[29]  Jacqueline Waniek,et al.  Cognitive Costs of Navigation Aids in Hypermedia Learning , 2008 .

[30]  Eric Jamet,et al.  Using video and static pictures to improve learning of procedural contents , 2009, Comput. Hum. Behav..

[31]  Brian P. Bailey,et al.  A framework for intelligent notification management in multitasking domains , 2008 .

[32]  Jason Thomas Bohan,et al.  Depth of processing and semantic anomalies , 2008 .

[33]  Fred Stentiford,et al.  Eye tracking — A new interface for visual exploration , 2006 .

[34]  Mary P. Kosarzycki,et al.  Emerging themes in distance learning research and practice: some food for thought , 2002 .

[35]  Hunter A. Murphy,et al.  3-D eye movement analysis , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[36]  Shumin Zhai,et al.  Chinese input with keyboard and eye-tracking: an anatomical study , 2001, CHI.

[37]  Poika Isokoski,et al.  Gazing and frowning as a new human--computer interaction technique , 2004, TAP.

[38]  Anne B. Sereno,et al.  Voluntary Spatial Attention has Different Effects on Voluntary and Reflexive Saccades , 2003, TheScientificWorldJournal.

[39]  Chris Berka,et al.  I-NET®: Interactive neuro-educational technology to accelerate skill learning , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[40]  E. Hess ATTITUDE AND PUPIL SIZE. , 1965, Scientific American.

[41]  F. Paas,et al.  Attention guidance in learning from a complex animation: Seeing is understanding? , 2010 .

[42]  Florian Schmidt-Weigand,et al.  A closer look at split visual attention in system- and self-paced instruction in multimedia learning , 2010 .

[43]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[44]  Ivar Bråten,et al.  Comprehension effects of signalling relationships between documents in search engines , 2010, Comput. Hum. Behav..

[45]  Eldon Todd Pierce Mental Workload Measurement Using the Intersaccadic Interval , 2009 .

[46]  Bryan Reimer,et al.  On-road driver eye movement tracking using head-mounted devices , 2002, ETRA.

[47]  Thierry Baccino,et al.  Approximation of on-line mental workload index in ATC simulated multitasks , 2010 .

[48]  Satoshi Kobori,et al.  Pupillary Responses during Learning of Inverted Tracking Tasks , 2009 .

[49]  Tsunhin John Wong,et al.  Capturing 'Aha!' moments of puzzle problems using pupillary responses and blinks , 2009 .

[50]  Brian P. Bailey,et al.  TAPRAV: An interactive analysis tool for exploring workload aligned to models of task execution , 2007, Interact. Comput..

[51]  Peter Brusilovsky,et al.  User-adaptive explanatory program visualization: evaluation and insights from eye movements , 2010, User Modeling and User-Adapted Interaction.

[52]  Peter Fröhlich,et al.  KIBITZER: a wearable system for eye-gaze-based mobile urban exploration , 2010, AH.

[53]  Yvonne Kammerer,et al.  Looking my way through the menu: the impact of menu design and multimodal input on gaze-based menu selection , 2008, ETRA.

[54]  Tao Lin,et al.  Using multiple data sources to get closer insights into user cost and task performance , 2008, Interact. Comput..

[55]  Sandra P. Marshall,et al.  Integrating psychophysiological measures of cognitive workload and eye movements to detect strategy shifts , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[56]  Dana Gaines Robinson,et al.  The Power of E-Performance. , 2001 .

[57]  Timo Partala,et al.  Affective Information in Human-Computer Interaction , 2009 .

[58]  Christina Merten On the use of eye-tracking in the assessment of self-explanation , 2006 .

[59]  Martin J. Steinbach,et al.  Mixed pro and antisaccade performance in children and adults , 2009, Brain Research.

[60]  Feng-Yang Kuo,et al.  An exploratory study of cognitive effort involved in decision under Framing - an application of the eye-tracking technology , 2009, Decis. Support Syst..

[61]  Victoria Johansson,et al.  Combined eyetracking and keystroke-logging methods for studying cognitive processes in text production , 2009, Behavior research methods.

[62]  Ying-Hua Guan,et al.  The effects of multimedia presentations on information processing : eye-movement analyses of text and picture integration in a multimedia-based learning scenario , 2002 .

[63]  B. Tversky,et al.  Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. , 2011, Psychophysiology.

[64]  André Vandierendonck,et al.  Input monitoring and response selection as components of executive control in pro-saccades and anti-saccades , 2007, Psychological research.

[65]  Marcia J. Simmering,et al.  E-learning: emerging uses, empirical results and future directions , 2003 .

[66]  Jan Noyes,et al.  Solving problems: How can guidance concerning task-relevancy be provided? , 2010, Comput. Hum. Behav..

[67]  Pia Knoeferle,et al.  Constituent order and semantic parallelism in online comprehension: Eye-tracking evidence from German , 2009, Quarterly journal of experimental psychology.

[68]  Jean-François Rouet,et al.  Understanding Multimedia Documents , 2010 .

[69]  Richard K. Lowe,et al.  An Eye Tracking Comparison of External Pointing Cues and Internal Continuous Cues in Learning with Complex Animations , 2010 .

[70]  Kasia Muldner,et al.  "Yes!": Using Tutor and Sensor Data to Predict Moments of Delight during Instructional Activities , 2010, UMAP.

[71]  Mark Witkowski,et al.  The inspection of very large images by eye-gaze control , 2008, AVI '08.

[72]  Richard J. Davidson,et al.  Individual differences in some (but not all) medial prefrontal regions reflect cognitive demand while regulating unpleasant emotion , 2009, NeuroImage.

[73]  Krista DeLeeuw When more cognitive load leads to less distraction , 2009 .

[74]  Cristina Conati,et al.  Eye-tracking for user modeling in exploratory learning environments: An empirical evaluation , 2007, Knowl. Based Syst..

[75]  Laurel A. King Visual Navigation Patterns and Cognitive Load , 2009, HCI.

[76]  E. Hess,et al.  Pupillometry: The Psychology of the Pupillary Response , 1978 .

[77]  Joseph T. Coyne,et al.  Applying Real Time Physiological Measures of Cognitive Load to Improve Training , 2009, HCI.

[78]  Josef C. Schrock,et al.  Working memory capacity and the antisaccade task: individual differences in voluntary saccade control. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[79]  Roel Vertegaal,et al.  ViewPointer: lightweight calibration-free eye tracking for ubiquitous handsfree deixis , 2005, UIST.

[80]  John C. Nesbit,et al.  How Students Read Concept Maps: A Study of Eye Movements , 2007 .

[81]  Alenoush Saroyan,et al.  Variations in lecturing styles , 1997 .

[82]  Jean Tower Using Computer Technology to Teach Thinking , 1996 .

[83]  Fred Stentiford,et al.  Eye Tracking: A New Interface for Visual Exploration , 2008, Multimodal Processing and Interaction.

[84]  Stephen J. Buckley,et al.  AN AUTOMOBILE-INTEGRATED SYSTEM FOR ASSESSING AND REACTING TO DRIVER COGNITIVE LOAD , 2002 .

[85]  Jeff Klingner,et al.  Fixation-aligned pupillary response averaging , 2010, ETRA.

[86]  E. Granholm,et al.  Pupillary responses on the visual backward masking task reflect general cognitive ability. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[87]  Florian Schmidt-Weigand,et al.  Explaining the modality and contiguity effects: New insights from investigating students' viewing behaviour , 2010 .

[88]  Ling Xia,et al.  Eye tracking and online search: Lessons learned and challenges ahead , 2008, J. Assoc. Inf. Sci. Technol..

[89]  Erkki Sutinen,et al.  Applying eye-movememt tracking to program visualization , 2005, 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC'05).

[90]  Jeffrey G. Morrison,et al.  Knowledge Desk Limited Objective Experiment (LOE) , 2003 .

[91]  Brian Murphy,et al.  The effects of gaze guiding on education software , 2007 .

[92]  E. Stuyven,et al.  The effect of cognitive load on saccadic eye movements. , 2000, Acta psychologica.

[93]  D. Teodorescu,et al.  The automatic and voluntary control of the scope of visual attention , 2004 .

[94]  Daniel Roggen,et al.  EyeMote - Towards Context-Aware Gaming Using Eye Movements Recorded from Wearable Electrooculography , 2008, Fun and Games.

[95]  Jacques D. Fleuriot,et al.  Diagrammatically-Driven Formal Verification of Web-Services Composition , 2012, Diagrams.

[96]  Oleg V. Komogortsev,et al.  An Effort Based Model of Software Usability , 2009 .

[97]  C. van Nimwegen,et al.  The paradox of the guided user: assistance can be counter-effective , 2008 .

[98]  Jens Alfredson,et al.  Verification of an Experimental Platform Integrating a Tobii Eyetracking System with the HiFi Game Engine , 2007 .

[99]  Yi Ting Huang,et al.  Online interpretation of scalar quantifiers: Insight into the semantics–pragmatics interface , 2009, Cognitive Psychology.

[100]  Xianjun Sam Zheng,et al.  Eyetracking in cognitive state detection for HCI , 2003, ICMI '03.

[101]  Glen R. Jarboe,et al.  Learning by doing: group projects in research methods classes , 1996 .

[102]  Weidong Huang,et al.  Beyond time and error: a cognitive approach to the evaluation of graph drawings , 2008, BELIV '08.

[103]  Chris Berka,et al.  Peak Performance Trainer (PPTTM): Interactive Neuro-educational Technology to Increase the Pace and Efficiency of Rifle Marksmanship Training , 2009, HCI.

[104]  T. Gog,et al.  Uncovering Expertise-Related Differences in Troubleshooting Performance: Combining Eye Movement and Concurrent Verbal Protocol Data , 2005 .

[105]  Frank Keller,et al.  Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure , 2010, ACL.

[106]  L. M. Silva,et al.  The construct of cognitive processing and speed : test performance and information processing approaches , 2009 .

[107]  D. Isaacowitz,et al.  Positive Gaze Preferences in Older Adults: Assessing the Role of Cognitive Effort with Pupil Dilation , 2010, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[108]  Mirjamaija Mikkilä-Erdmann,et al.  Constructing Mental Models during Learning from Science Text , 2008 .

[109]  Stephen Doherty,et al.  Eye tracking as an MT evaluation technique , 2010, Machine Translation.

[110]  Özge Alaçam,et al.  A Usability Study of WebMaps with Eye Tracking Tool: The Effects of Iconic Representation of Information , 2009, HCI.

[111]  Andreas Paepcke,et al.  Gaze-enhanced user interface design , 2007 .

[112]  Pierre Dillenbourg,et al.  Display of Key Pictures from Animation: Effects on Learning , 2008 .

[113]  Minoru Nakayama,et al.  Attentive interfaces for users with disabilities: eye gaze for intention and uncertainty estimation , 2009, Universal Access in the Information Society.

[114]  Robert Rohling,et al.  An application of eyegaze tracking for designing radiologists' workstations: Insights for comparative visual search tasks , 2006, TAP.

[115]  Nicholas P. Murray,et al.  AN ASSESSMENT OF THE EFFICIENCY AND EFFECTIVENESS OF SIMULATED AUTO RACING PERFORMANCE: PSYCHOPHYSIOLOGICAL EVIDENCE FOR THE PROCESSING EFFICIENCY THEORY AS INDEXED THROUGH VISUAL SEARCH CHARACTERISTICS AND P300 RECIPROCITY , 2000 .

[116]  Robert J. K. Jacob,et al.  Evaluation of eye gaze interaction , 2000, CHI.

[117]  J. Andreassi Psychophysiology: Human Behavior and Physiological Response , 1980 .

[118]  Naini H. Mistry Looking Beyond the Eye: Pupillary Dilation and its Relationship to Conversational Behavior in Computer-mediated Conversation and Face-to-Face Conversation , 2005 .

[119]  Andreas Paepcke,et al.  EyePoint: practical pointing and selection using gaze and keyboard , 2007, CHI.

[120]  Jess Gilland Driving, eye -tracking and visual entropy: Exploration of age and task effects , 2008 .

[121]  K. Holmqvist,et al.  Reading Information Graphics: the Role of Spatial Contiguity and Dual Attentional Guidance , 2022 .

[122]  Judi McCuaig,et al.  Detecting Learner Frustration: Towards Mainstream Use Cases , 2010, Intelligent Tutoring Systems.

[123]  Eric N. Wiebe,et al.  Visual Representations of DNA Replication: Middle Grades Students’ Perceptions and Interpretations , 2005 .

[124]  Zhiwei Guan,et al.  The validity of the stimulated retrospective think-aloud method as measured by eye tracking , 2006, CHI.

[125]  John Sweller,et al.  Cognitive Load Theory: Instructional Implications of the Interaction between Information Structures and Cognitive Architecture , 2004 .

[126]  Nemanja Memarovic The influence of personal navigation devices on drivers' visual attention on the road ahead and driving performance , 2009 .

[127]  Keith Davids Visual search strategies , 2006 .

[128]  Colin Camerer,et al.  Pinocchio's Pupil: Using Eyetracking and Pupil Dilation to Understand Truth-Telling and Deception in Sender-Receiver Game , 2009 .

[129]  Addie Ehrenstein,et al.  Training for a rapidly changing workplace : applications of psychological research , 1997 .

[130]  S. P. Marshall,et al.  The Index of Cognitive Activity: measuring cognitive workload , 2002, Proceedings of the IEEE 7th Conference on Human Factors and Power Plants.

[131]  Marco Porta,et al.  e5Learning, an E-Learning Environment Based on Eye Tracking , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[132]  John D. Lee,et al.  Real-Time Detection of Driver Cognitive Distraction Using Support Vector Machines , 2007, IEEE Transactions on Intelligent Transportation Systems.

[133]  Han-Chin Liu,et al.  An examination of cognitive processing of multimedia information based on viewers' eye movements , 2011, Interact. Learn. Environ..

[134]  Donald L. Fisher,et al.  Driver-Eye-Movement-Based Investigation for Improving Work-Zone Safety , 2009 .

[135]  Robert E. Kraut,et al.  Controlling interruptions: awareness displays and social motivation for coordination , 2004, CSCW.

[136]  E. Mory Feedback research revisited. , 2004 .

[137]  Wolfgang Schnotz,et al.  Effects of animation's speed of presentation on perceptual processing and learning , 2010 .

[138]  John Paulin Hansen,et al.  Single gaze gestures , 2010, ETRA '10.

[139]  Helen Joanna Arabella Fuller,et al.  The Virtual Driver : integrating physical and cognitive human models to simulate driving with a secondary in-vehicle task , 2010 .

[140]  Manfred Tscheligi,et al.  Visual Search Strategies of Tag Clouds - Results from an Eyetracking Study , 2009, INTERACT.

[141]  Hsiao-Ching She,et al.  The impact of multimedia effect on science learning: Evidence from eye movements , 2009, Comput. Educ..

[142]  Rolf Nordahl,et al.  Distraction as a measure of presence: using visual and tactile adjustable distraction as a measure to determine immersive presence of content in mediated environments , 2010, Virtual Reality.

[143]  Sue R. Crull,et al.  Adapting Traditions: Teaching Research Methods in a Large Class Setting , 2004 .

[144]  Christof Körner,et al.  Sequential processing in comprehension of hierarchical graphs , 2004 .

[145]  M. Traxler A Hierarchical Linear Modeling Analysis of Working Memory and Implicit Prosody in the Resolution of Adjunct Attachment Ambiguity , 2009, Journal of psycholinguistic research.

[146]  J. Adam,et al.  Cue validity effects in response preparation: A pupillometric study , 2008, Brain Research.

[147]  Andreas Dengel,et al.  Query expansion using gaze-based feedback on the subdocument level , 2008, SIGIR '08.

[148]  Eric N. Wiebe,et al.  Eye-Tracking Students' Attention to PowerPoint Photographs in a Science Education Setting , 2005 .

[149]  E. Wiebe,et al.  The influence of prior knowledge on viewing and interpreting graphics with macroscopic and molecular representations , 2008 .

[150]  Michael G. Hardin,et al.  Age-related influence of contingencies on a saccade task , 2006, Experimental Brain Research.

[151]  Brian S. Schnitzer,et al.  Eye movements during multiple readings of the same text , 2006, Vision Research.

[152]  Fred Stentiford,et al.  A performance comparison of eye tracking and mouse interfaces in a target image identification task , 2005 .

[153]  Martha E. Crosby,et al.  Assessing Cognitive Load with Physiological Sensors , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[154]  Adam Feil Attention and encoding in physics learning and problem solving , 2009 .