Automated Eye-Movement Protocol Analysis

This article describes and evaluates a class of methods for performing automated analysis of eye-movement protocols. Although eye movements have become increasingly popular as a tool for investigating user behavior, they can be extremely difficult and tedious to analyze. In this article we propose an approach to automating eye-movement protocol analysis by means of tracing-relating observed eye movements to the sequential predictions of a process model. We present three tracing methods that provide fast and robust analysis and alleviate the equipment noise and individual variability prevalent in typical eye-movement protocols. We also describe three applications of the tracing methods that demonstrate how the methods facilitate the use of eye movements in the study of user behavior and the inference of user intentions.

[1]  Frank J. Lee,et al.  Does Learning a Complex Task Have to Be Complex?: A Study in Learning Decomposition , 2001, Cognitive Psychology.

[2]  Eileen Kowler The role of visual and cognitive processes in the control of eye movement. , 1990, Reviews of oculomotor research.

[3]  Joseph H. Goldberg,et al.  Visual Search of Food Nutrition Labels , 1999, Hum. Factors.

[4]  Gerald L. Lohse,et al.  Organizational Behavior and Human Decision Processes a Comparison of Two Process Tracing Methods for Choice Tasks , 2022 .

[5]  John R. Anderson,et al.  Eye movements and algebra tutoring , 1999 .

[6]  L. Young,et al.  Survey of eye movement recording methods , 1975 .

[7]  K. Rayner Eye Movements and Cognitive Processes in Reading, Visual Search, and Scene Perception , 1995 .

[8]  Aulikki Hyrskykari,et al.  101 spots, or how do users read menus? , 1998, CHI.

[9]  R. Groner Eye Movements and Psychological Functions: International Views , 1983 .

[10]  Yannick Lallement A Hierarchical Ensemble of Decision Trees Applied to Classifying Data from a Psychological Experiment , 1998, FLAIRS Conference.

[11]  P. Viviani Eye movements in visual search: cognitive, perceptual and motor control aspects. , 1990, Reviews of oculomotor research.

[12]  P Suppes,et al.  Eye-movement models for arithmetic and reading performance. , 1990, Reviews of oculomotor research.

[13]  Patrick Suppes,et al.  A procedural theory of eye movements in doing arithmetic , 1983 .

[14]  J. Stern Theoretical and applied aspects of eye movement research A. G. Gale and F. Johnson, (Elsevier Science Publishers B.V., Amsterdam, 1984) pp. xiii + 565, Dfl. 185 , 1985, Biological Psychology.

[15]  Allen Newell,et al.  PAS-II: An Interactive Task-Free Version of an Automatic Protocol Analysis System , 1973, IEEE Transactions on Computers.

[16]  Holly A. Yanco,et al.  Wheelesley: A Robotic Wheelchair System: Indoor Navigation and User Interface , 1998, Assistive Technology and Artificial Intelligence.

[17]  M A Just,et al.  A theory of reading: from eye fixations to comprehension. , 1980, Psychological review.

[18]  J. Kruskal An Overview of Sequence Comparison: Time Warps, String Edits, and Macromolecules , 1983 .

[19]  John R. Anderson,et al.  Eye tracking the visual search of click-down menus , 1999, CHI '99.

[20]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[21]  G. Legge,et al.  Mr. Chips: an ideal-observer model of reading. , 1997, Psychological review.

[22]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[23]  Gerald L. Lohse,et al.  Consumer Eye Movement Patterns on Yellow Pages Advertising , 1997 .

[24]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[25]  Thomas Sudkamp,et al.  Languages and Machines , 1988 .

[26]  W. W. Peterson,et al.  Using Eye Movements to Classify Search Strategies , 1991 .

[27]  I. S. Mackenzie,et al.  Virtual Environments and Advanced Interface Design , 1995 .

[28]  John R. Anderson,et al.  Mapping eye movements to cognitive processes , 1999 .

[29]  John R. Anderson,et al.  Integrating analogical mapping and general problem solving: the path-mapping theory , 2001, Cogn. Sci..

[30]  Tayyar Sen,et al.  The Effects of Task Variables and Prolonged Performance on Saccadic Eye Movement Parameters , 1984 .

[31]  Gregory W. Edwards A tool for creating eye-aware applications that adapt to changes in user behaviors , 1998, Assets '98.

[32]  Tom Johnston,et al.  MacSHAPA and the enterprise of exploratory sequential data analysis (ESDA) , 1994, Int. J. Hum. Comput. Stud..

[33]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[34]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[35]  Eyal M. Reingold,et al.  Selection By Looking: A Novel Computer Interface And Its Application To Psychological Research , 1995 .

[36]  Erik D. Reichle,et al.  Toward a model of eye movement control in reading. , 1998, Psychological review.

[37]  Bruce Lowerre,et al.  The Harpy speech understanding system , 1990 .

[38]  Rudolf Groner,et al.  From eye to mind : information acquisition in perception, search, and reading , 1990 .

[39]  L. Stark,et al.  Scanpaths in saccadic eye movements while viewing and recognizing patterns. , 1971, Vision research.

[40]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[41]  Casper J. Erkelens,et al.  The Initial Direction and Landing Position of Saccades , 1995 .

[42]  Joseph H. Goldberg,et al.  Eye-gaze determination of user intent at the computer interface , 1993 .

[43]  Frank E. Ritter,et al.  Developing Process Models as Summaries of HCI Action Sequences , 1994, Hum. Comput. Interact..

[44]  Dario D. Salvucci Inferring intent in eye-based interfaces: tracing eye movements with process models , 1999, CHI '99.

[45]  Worthy N. Martin,et al.  Human-computer interaction using eye-gaze input , 1989, IEEE Trans. Syst. Man Cybern..

[46]  John R. Anderson,et al.  Intelligent gaze-added interfaces , 2000, CHI.

[47]  Frank E. Ritter A Methodology and Software Environment for Testing Process Model's Sequential Predictions with Protocols , 1992 .

[48]  Susanne Trauzettel-Klosinski,et al.  From Eye to Mind: Information Acquisition in Perception, Search, and Reading , 1997 .

[49]  John R. Anderson,et al.  Tracing Eye Movement Protocols with Cognitive Process Models , 1998 .

[50]  H. Widdel Operational Problems in Analysing Eye Movements , 1984 .

[51]  M. Hegarty,et al.  Comprehension of Arithmetic Word Problems: Evidence from Students' Eye Fixations. , 1992 .

[52]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[53]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[54]  James Gips,et al.  On Building Intelligence into EagleEyes , 1998, Assistive Technology and Artificial Intelligence.

[55]  Eileen Kowler,et al.  The effect of expectations on slow oculomotor control—IV. Anticipatory smooth eye movements depend on prior target motions , 1984, Vision Research.

[56]  Herbert A. Simon,et al.  Problem Solving in Semantically Rich Domains: An Example from Engineering Thermodynamics , 1977, Cogn. Sci..

[57]  Dana K. Smith,et al.  Automated Protocol Analysis , 1993, Hum. Comput. Interact..

[58]  Allen Newell,et al.  Protocol Analysis as a Task for Artificial Intelligence , 1971, IJCAI.

[59]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[60]  G. McConkie,et al.  Asymmetry of the perceptual span in reading , 1976 .

[61]  S. Ohlsson Trace Analysis and Spatial Reasoning: An Example of Intensive Cognitive Diagnosis and Its Implications for Testing. September 1987. Technical Report. , 1987 .

[62]  K. Rayner,et al.  Comparing naming, lexical decision, and eye fixation times: Word frequency effects and individual differences , 1998, Memory & cognition.

[63]  D. F. Fisher,et al.  Eye movements : cognition and visual perception , 1982 .

[64]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[65]  M. Just,et al.  From the SelectedWorks of Marcel Adam Just 1992 A capacity theory of comprehension : Individual differences in working memory , 2017 .

[66]  Christopher M. Harris,et al.  The distribution of fixation durations in infants and naive adults , 1988, Vision Research.

[67]  Bryn Wolfe,et al.  A Neural Network Approach to Tracking Eye Position , 1997, Int. J. Hum. Comput. Interact..

[68]  Robert J. K. Jacob,et al.  Eye tracking in advanced interface design , 1995 .

[69]  Shumin Zhai,et al.  Manual and gaze input cascaded (MAGIC) pointing , 1999, CHI '99.

[70]  Bonnie E. John,et al.  Browser-Soar: a computational model of a highly interactive task , 1992, CHI.

[71]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[72]  K. Preston White,et al.  Eye-gaze word processing , 1990, IEEE Trans. Syst. Man Cybern..

[73]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[74]  Kurt VanLehn,et al.  Cirrus: An Automated Protocol Analysis Tool. , 1987 .