Learning Image-Derived Eye Movement Patterns to Characterize Perceptual Expertise

Learning Image-Derived Eye Movement Patterns to Characterize Perceptual Expertise Rui Li (rxl5604@rit.edu) College of Computing and Information Science, 1 Lomb Memorial Drive Rochester, NY 14623 USA Jeff Pelz (pelz@cis.rit.edu) College of Imaging Science, 1 Lomb Memorial Drive Rochester, NY 14623 USA Pengcheng Shi (spcast@rit.edu) Anne R. Haake (arhics@rit.edu) College of Computing and Information Science, 1 Lomb Memorial Drive Rochester, NY 14623 USA Abstract Experts have remarkable capability of locating, identifying and categorizing objects in their domain-specific images. Eliciting experts’ visual strategies will benefit image understanding by transferring human domain knowledge into image-based com- putational procedures. In this paper, an experiment conducted to collect both eye movement and verbal description data from three groups of subjects with different medical training lev- els (eleven board-certified dermatologists, four dermatologists in training and thirteen novices) while they were examining and describing 42 photographic dermatological images. We present a hierarchical probabilistic framework to discover the stereotypical and idiosyncratic viewing behaviors exhibited within each group when they are diagnosing medical images. Furthermore, experts’ annotations of thought units on the tran- scribed verbal descriptions are time-aligned with discovered eye movement patterns to interpret their semantic meanings. By mapping eye movement patterns to thought units, we un- cover the manner in which these subjects alternated their be- haviors over the course of inspection and how these experts parse the images. Keywords: Eye movements; eye tracking; verbal description; multimodal data analysis; graphical model; user study; diag- nostic reasoning Introduction Perceptual expertise is considered to be the crucial cogni- tive factor accounting for the advantage of highly trained ex- perts (Hoffman & Fiore, 2007). Experts generate distinc- tively different perceptual representations when they view the same scene as novices (Palmeri, Wong, & Gauthier, 2004 ; Smuc, Mayr, & Windhager, 2010). Rather than passively ”photocopying” the visual information directly from sensors into minds, visual perception actively interprets the infor- mation by altering perceptual representations of the images based on experience and goals. By analyzing the whole se- quences of fixation and saccadic eye movements from groups with different expertise levels, significant differences in vi- sual search strategies between groups show human exper- tise plays a great role in medical image examination. In (Manning, Ethell, Donovan, & Crawford, 2006), the nature of expert performance of four observer groups with different levels of expertise was investigated . They compared multi- ple eye movement measures and suggested these distinctive variations among the observations of the better performance from higher expertise level are due to the consequences of experience and training. In (Krupinski et al., 2006), an eye movement study was conducted on diagnostic pathology of light microscopy to identify distinctive viewing stereotypes for each level of experience . Their results suggest eye move- ment monitoring could serve as a basis for the creation of innovative pathology training routines. In knowledge-rich domains, perceptual expertise is par- ticularly valuable. Medical image understanding via manu- ally marking and annotating become not only labor intensive for experts but also ineffective because of the variability and noise of experts’ performance (Gordon, Lotenberg, Jeronimo, & Greenspan, 2009). For training and designing decision support systems, the basic perceptual strategies and principles of diagnostic-reasoning are also desired (Dempere-Marco, Hu, & Yang, 2011). To address this problem, it requires the ability of extracting and representing experts’ perceptual ex- pertise in a form that is ready to be applied. In this work, our contributions are: first, we discover and represent expertise- related eye movement patterns exhibited among multiple ex- perts in an objective and unbiased way; second, to validate these patterns, we identify their semantic meanings by time- aligning them with standardized thought units annotated by additional experts. Third, we also characterize the eye move- ment patterns of three different expertise levels respectively which can be used to categorize users’ expertise levels based on their visual inspection on medical images. Human viewing behaviors are valuable yet effortless re- sources worth of exploiting. In specific domains experts per- ceptual expertise is considered to be more consistent and in- formative than their manual markings. Human vision is an active dynamic process in which the viewer seeks out spe- cific information to support ongoing cognitive and behav- ioral activity (Henderson & Malcolm, 2009). Since visual acuity is limited to the foveal region and resolution fades dramatically in the periphery, we move our eyes to bring a portion of the visual field into high resolution at the center of gaze. Studies have shown that visual attention is influ-