This thesis investigated the efficacy of eye tracking technology and a digital training tool in radiographic image interpretation. A systematic review was performed to investigate the performance of reporting radiographers completing chest image interpretation following training. The quality of evidence published in this area was high. The role of image interpretation differed between studies, ranging between: image red dot abnormality highlighting, image comment and clinical reporting. A comparison of image interpretation skills of radiographers across a range of experience was completed using eye tracking technology. Reporting radiographers trained in MSK image interpretation demonstrated statistically significant accuracy rates (p≤0.001), and confidence levels (p≤0.001) and took a mean of 2.4 s longer to clinically decide on an image compared to students. Reporting radiographers also had a statistically greater accuracy rate (p≤0.001), were more confident (p≤0.001) and took longer to clinically decide (14 s on average) on an image diagnosis (p=0.02) than radiographers. Eye tracking patterns presented within heat maps, were a good reflection of group expertise and search strategies. Eye tracking metrics were indicative of participant performance and reflected the different search strategies that each group of participants adopted during their image interpretations. A digital training tool for use in chest image interpretation was created based on evidence within the literature, using expert input and two search strategies previously used in clinical practice. Images and diagrams, aiding translation of the tool content, were incorporated where possible. Improvements were seen in interpretation performance and confidence (p<0.05). There was a decrease in FP values and increase in TN values seen in the intervention group (p<0.05). This tool therefore has the potential to be used as a training tool in chest image interpretation for reporting clinicians and healthcare professionals. This work may contribute to improving diagnosis and help reduce reporting times.

This paper describes the possibility of monitoring the progress of knowledge and skills acquisition by the students of radiology. It is achieved by an analysis of a visual attention distribution patterns during image-based tasks solving. The concept is to use the eye-tracking data to recognize the way how the radiographic images are read by recognized experts, radiography residents involved in the training program, and untrained users who graduated from biomedical engineering. The results of research presented in this paper support the usefulness of earlier elaborated eye metrics, Visual Attention Time Product VATP, for skills measurement when using Electronic Medical Record for CT images evaluation. Moreover, it appears that it is possible to differentiate the level of residents skills in radiology structures recognition. Presented methodology can also be utilized in other areas where operator performs image related tasks.

Several computational visual saliency models have been proposed in the context of viewing natural scenes. We aim to investigate the relevance of computational saliency models in medical images in the context of abnormality detection. We report on two studies aimed at understanding the role of visual saliency in medical images. Diffuse lesions in Chest X-Ray images, which are characteristic of Pneumoconiosis and high contrast lesions such as 'Hard Exudates' in retinal images were chosen for the study. These approximately correspond to conjunctive and disjunctive targets in a visual search task. Saliency maps were computed using three popular models namely Itti-Koch [7], GBVS [3] and SR [4]. The obtained maps were evaluated against gaze maps and ground truth from medical experts. Our results show that GBVS is seen to perform the best (Mdn. ROC area = 0.77) for chest X-Ray images while SR performs the best (ROC area = 0.73) for retinal images, thus asserting that searching for conjunctive targets calls for a more local examination of an image while disjunctive targets call for a global examination. Based on the results of the above study, we propose extensions for the two best performing models. The first extension makes use of top down knowledge such as lung segmentation. This is shown to improve the performance of GBVS to some extent. In the second case the extension is by way of including multi-scale information. This is shown to significantly (by 28.76%) improve abnormality detection. The key insight from these studies is that bottom saliency continues to play a predominant role in examining medical images.

Physician training has greatly benefitted from insights gained in understanding the manner in which experts search medical images for abnormalities. The aims of this study were to compare the search patterns of 30 fourth-year dental students and 15 certified oral and maxillofacial radiologists (OMRs) over panoramic images and to determine the most robust variables for future studies involving image visualization. Eye tracking was used to capture the eye movement patterns of both subject groups when examining 20 panoramic images classified as normal or abnormal. Abnormal images were further subclassified as having an obvious, intermediate, or subtle abnormality. The images were presented in random order to each participant, and data were collected on duration of the participants' observations and total distance tracked, time to first eye fixation, and total duration and numbers of fixations on and off the area of interest (AOI). The results showed that the OMRs covered greater distances than the dental students (p<0.001) for normal images. For images of pathosis, the OMRs required less total time (p<0.001), made fewer eye fixations (p<0.01) with fewer saccades (p<0.001) than the students, and required less time before making the first fixation on the AOI (p<0.01). Furthermore, the OMRs covered less distance (p<0.001) than the dental students for obvious pathoses. For investigations of images of pathosis, time to first fixation is a robust parameter in predicting ability. For images with different levels of subtlety of pathoses, the number of fixations, total time spent, and numbers of revisits are important parameters to analyze when comparing observer groups with different levels of experience.

PURPOSE To determine the effectiveness of radiologists' search, recognition, and acceptance of lung nodules on computed tomographic (CT) images by using eye tracking. MATERIALS AND METHODS This study was performed with a protocol approved by the institutional review board. All study subjects provided informed consent, and all private health information was protected in accordance with HIPAA. A remote eye tracker was used to record time-varying gaze paths while 13 radiologists interpreted 40 lung CT images with an average of 3.9 synthetic nodules (5-mm diameter) embedded randomly in the lung parenchyma. The radiologists' gaze volumes ( GV gaze volume s) were defined as the portion of the lung parenchyma within 50 pixels (approximately 3 cm) of all gaze points. The fraction of the total lung volume encompassed within the GV gaze volume s, the fraction of lung nodules encompassed within each GV gaze volume (search effectiveness), the fraction of lung nodules within the GV gaze volume detected by the reader (recognition-acceptance effectiveness), and overall sensitivity of lung nodule detection were measured. RESULTS Detected nodules were within 50 pixels of the nearest gaze point for 990 of 992 correct detections. On average, radiologists searched 26.7% of the lung parenchyma in 3 minutes and 16 seconds and encompassed between 86 and 143 of 157 nodules within their GV gaze volume s. Once encompassed within their GV gaze volume , the average sensitivity of nodule recognition and acceptance ranged from 47 of 100 nodules to 103 of 124 nodules (sensitivity, 0.47-0.82). Overall sensitivity ranged from 47 to 114 of 157 nodules (sensitivity, 0.30-0.73) and showed moderate correlation (r = 0.62, P = .02) with the fraction of lung volume searched. CONCLUSION Relationships between reader search, recognition and acceptance, and overall lung nodule detection rate can be studied with eye tracking. Radiologists appear to actively search less than half of the lung parenchyma, with substantial interreader variation in volume searched, fraction of nodules included within the search volume, sensitivity for nodules within the search volume, and overall detection rate.

Our prior visual experience plays a critical role in face perception. We show superior perceptual performance for differentiating conspecific (vs non-conspecific), own-race (vs other-race) and familiar (vs unfamiliar) faces. However, it remains unclear whether our experience with faces of other species would influence our gaze allocation for extracting salient facial information. In this eye-tracking study, we asked both dog owners and non-owners to judge the approachability of human, monkey and dog faces, and systematically compared their behavioural performance and gaze pattern associated with the task. Compared to non-owners, dog owners assessed dog faces with shorter time and fewer fixations, but gave higher approachability ratings. The gaze allocation within local facial features was also modulated by the ownership. The averaged proportion of the fixations and viewing time directed at the dog mouth region were significantly less for the dog owners, and more experienced dog owners tended to look more at the dog eyes, suggesting the adoption of a prior experience-based viewing behaviour for assessing dog approachability. No differences in behavioural performance and gaze pattern were observed between dog owners and non-owners when judging human and monkey faces, implying that the dog owner’s experience-based gaze strategy for viewing dog faces was not transferable across faces of other species.

Objective: The aim of this study was to explore reader gaze, performance, and preference during interpretation of cranial computed tomography (cCT) in stack mode at two different sizes. Background: Digital display of medical images allows for the manipulation of many imaging factors, like image size, by the radiologists, yet it is often not known what display parameters better suit human perception. Method: Twenty-one radiologists provided informed consent to be eye tracked while reading 20 cCT cases. Half of these cases were presented at a size of 14 × 14 cm (512 × 512 pixels), half at 28 × 28 cm (1,024 × 1,024 pixels). Visual search, performance, and preference for the two image sizes were assessed. Results: When reading small images, significantly fewer, but longer, fixations were observed, and these fixations covered significantly more slices. Time to first fixation of true positive findings was faster in small images, but dwell time on true findings was longer. Readers made more false positive decisions in small images, but no overall difference in either jackknife alternative free-response receiver operating characteristic or reading time was found. Conclusion: Overall performance is not affected by image size. However, small-stack-mode cCT images may better support the use of motion perception and acquiring an overview, whereas large-stack-mode cCT images seem better suited for detailed analyses. Application: Subjective and eye-tracking data suggest that image size influences how images are searched and that different search strategies might be beneficial under different circumstances.

OBJECTIVES The assessment of digital ulcers (DUs) in systemic sclerosis (SSc) depends crucially on visual aspects. However, little is known about the differences in visual assessment of these wounds between experts and non-experts or medical lay persons (novices). To develop potential training recommendations for the visual assessment of digital ulcers in SSc, we analysed gaze pattern data during assessment of digital ulcers between assessors with different levels of expertise. METHODS Gaze pattern data from 36 participants were collected with a mobile eye tracking device. 20 pictures from digital ulcers of SSc patients were presented to each participant. The analysis comprised the scan path, the dwell times (for areas of interest), fixation counts and the entry time for each picture and subject. RESULTS Most significant differences were found between novices and medically educated groups. Dwell times in the wound area for novices differed statistically significantly from all medically educated groups (1.76s-3.1s less). Above all, novices had lower dwell times in wound margin and wound surrounding and spent more time in other areas (up to 1.92s longer). Correspondingly, they had less fixation points and longer overall scan paths in wound areas. Similar gaze pattern data were generated for medically educated groups. CONCLUSIONS For the first time, we could analyse the visual assessment of digital ulcers in SSc and detected differences according to levels of medical education and expertise. Adequate training on proper interpretation of changes in all wound areas are warranted to improve wound assessment in digital ulcers.

OBJECTIVE Eye tracking in three dimensions is novel, but established descriptors derived from two-dimensional (2D) studies are not transferable. We aimed to develop metrics suitable for statistical comparison of eye-tracking data obtained from readers of three-dimensional (3D) "virtual" medical imaging, using CT colonography (CTC) as a typical example. METHODS Ten experienced radiologists were eye tracked while observing eight 3D endoluminal CTC videos. Subsequently, we developed metrics that described their visual search patterns based on concepts derived from 2D gaze studies. Statistical methods were developed to allow analysis of the metrics. RESULTS Eye tracking was possible for all readers. Visual dwell on the moving region of interest (ROI) was defined as pursuit of the moving object across multiple frames. Using this concept of pursuit, five categories of metrics were defined that allowed characterization of reader gaze behaviour. These were time to first pursuit, identification and assessment time, pursuit duration, ROI size and pursuit frequency. Additional subcategories allowed us to further characterize visual search between readers in the test population. CONCLUSION We propose metrics for the characterization of visual search of 3D moving medical images. These metrics can be used to compare readers' visual search patterns and provide a reproducible framework for the analysis of gaze tracking in the 3D environment. ADVANCES IN KNOWLEDGE This article describes a novel set of metrics that can be used to describe gaze behaviour when eye tracking readers during interpretation of 3D medical images. These metrics build on those established for 2D eye tracking and are applicable to increasingly common 3D medical image displays.

New advances in Medical imaging have made a great impact on diagnosis and treatment planning. The most important part of medical image processing is the identifi cation of reliable and accurate ROIs. The technological advances in the fi eld of medical image processing require that the capabilities of the human perceptual system are taken into account for identifi cation and extraction of ROI. In general, the main objective of these approaches is to achieve higher-accuracy rate and lower computational costs. This study summarizes the visual attention modeling and discusses the various available visual attention modals used for ROI extraction in medical image processing.

Machine learning is a powerful and effective tool for medical image analysis to perform computer-aided diagnosis (CAD). Having great potential in improving the accuracy of a diagnosis, CAD systems are often analyzed in terms of the final accuracy, leading to a limited understanding of the internal decision process, impossibility to gain insights, and ultimately to skepticism from clinicians. We present a visual analysis approach to uncover the decision-making process of a CAD system for classifying pancreatic cystic lesions. This CAD algorithm consists of two distinct components: random forest (RF), which classifies a set of predefined features, including demographic features, and a convolutional neural network (CNN), which analyzes radiological features of the lesions. We study the class probabilities generated by the RF and the semantical meaning of the features learned by the CNN. We also use an eye tracker to better understand which radiological features are particularly useful for a radiologist to make a diagnosis and to quantitatively compare with the features that lead the CNN to its final classification decision. Additionally, we evaluate the effects and benefits of supplying the CAD system with a case-based "visual aid" in a second-reader setting.

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

In this paper we present GroupGaze. It is a tool that can be used to calculate the gaze direction and the gaze position of whole groups. GroupGazer calculates the gaze direction of every single person in the image and allows to map these gaze vectors to a projection like a projector. In addition to the person-specific gaze direction, the person affiliation of each gaze vector is stored based on the position in the image. Also, it is possible to save the group attention after a calibration. The software is free to use and requires a simple webcam as well as an NVIDIA GPU and the operating system Windows or Linux. Link: https://es-cloud.cs.uni-tuebingen.de/d/8e2ab8c3fdd444e1a135/?p=%2FGroupGazer&mode=list

In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise.

INTRODUCTION It is well known that accurate interpretation of the 12-lead electrocardiogram (ECG) requires a high degree of skill. There is also a moderate degree of variability among those who interpret the ECG. While this is the case, there are no best practice guidelines for the actual ECG interpretation process. Hence, this study adopts computerized eye tracking technology to investigate whether eye-gaze can be used to gain a deeper insight into how expert annotators interpret the ECG. Annotators were recruited in San Jose, California at the 2013 International Society of Computerised Electrocardiology (ISCE). METHODS Each annotator was recruited to interpret a number of 12-lead ECGs (N=12) while their eye gaze was recorded using a Tobii X60 eye tracker. The device is based on corneal reflection and is non-intrusive. With a sampling rate of 60Hz, eye gaze coordinates were acquired every 16.7ms. Fixations were determined using a predefined computerized classification algorithm, which was then used to generate heat maps of where the annotators looked. The ECGs used in this study form four groups (3=ST elevation myocardial infarction [STEMI], 3=hypertrophy, 3=arrhythmias and 3=exhibiting unique artefacts). There was also an equal distribution of difficulty levels (3=easy to interpret, 3=average and 3=difficult). ECGs were displayed using the 4x3+1 display format and computerized annotations were concealed. RESULTS Precisely 252 expert ECG interpretations (21 annotators×12 ECGs) were recorded. Average duration for ECG interpretation was 58s (SD=23). Fleiss' generalized kappa coefficient (Pa=0.56) indicated a moderate inter-rater reliability among the annotators. There was a 79% inter-rater agreement for STEMI cases, 71% agreement for arrhythmia cases, 65% for the lead misplacement and dextrocardia cases and only 37% agreement for the hypertrophy cases. In analyzing the total fixation duration, it was found that on average annotators study lead V1 the most (4.29s), followed by leads V2 (3.83s), the rhythm strip (3.47s), II (2.74s), V3 (2.63s), I (2.53s), aVL (2.45s), V5 (2.27s), aVF (1.74s), aVR (1.63s), V6 (1.39s), III (1.32s) and V4 (1.19s). It was also found that on average the annotator spends an equal amount of time studying leads in the frontal plane (15.89s) when compared to leads in the transverse plane (15.70s). It was found that on average the annotators fixated on lead I first followed by leads V2, aVL, V1, II, aVR, V3, rhythm strip, III, aVF, V5, V4 and V6. We found a strong correlation (r=0.67) between time to first fixation on a lead and the total fixation duration on each lead. This indicates that leads studied first are studied the longest. There was a weak negative correlation between duration and accuracy (r=-0.2) and a strong correlation between age and accuracy (r=0.67). CONCLUSIONS Eye tracking facilitated a deeper insight into how expert annotators interpret the 12-lead ECG. As a result, the authors recommend ECG annotators to adopt an initial first impression/pattern recognition approach followed by a conventional systematic protocol to ECG interpretation. This recommendation is based on observing misdiagnoses given due to first impression only. In summary, this research presents eye gaze results from expert ECG annotators and provides scope for future work that involves exploiting computerized eye tracking technology to further the science of ECG interpretation.

Human observers with varying degrees of expertise interpret the 12-lead electrocardiogram (ECG) in different ways. In this work the authors investigate whether eye tracking can be used to gain an insight into how a human observer interprets the 12-lead ECG. A clinical scientist interpreted 29 ECGs whilst an eye tracking device was used to record eye movement patterns. The time dedicated to studying each lead across all 29 ECGs was recorded. The observer dedicated 26% of his time to studying the rhythm strip and only 1% of his time to studying lead aVR (t-test: p-value <; 0.01). Eye tracking has the potential to provide an insight into how an individual observer interprets an ECG.

Gaze tracking is a promising technology for studying the visual perception of clinicians during image-based medical exams. It could be used in longitudinal studies to analyze their perceptive process, explore human-machine interactions, and develop innovative computer-aided imaging systems. However, using a remote eye tracker in an unconstrained environment and over time periods of weeks requires a certain guarantee of performance to ensure that collected gaze data are fit for purpose. We report the results of evaluating eye tracking calibration for longitudinal studies. First, we tested the performance of an eye tracker on a cohort of 13 users over a period of one month. For each participant, the eye tracker was calibrated during the first session. The participants were asked to sit in front of a monitor equipped with the eye tracker, but their position was not constrained. Second, we tested the performance of the eye tracker on sonographers positioned in front of a cart-based ultrasound scanner. Experimental results show a decrease of accuracy between calibration and later testing of 0.30° and a further degradation over time at a rate of 0.13°. month−1. The overall median accuracy was 1.00° (50.9 pixels) and the overall median precision was 0.16° (8.3 pixels). The results from the ultrasonography setting show a decrease of accuracy of 0.16° between calibration and later testing. This slow degradation of gaze tracking accuracy could impact the data quality in long-term studies. Therefore, the results we present here can help in planning such long-term gaze tracking studies.

Eye-tracking—the process of measuring where people look in a visual field—has been widely used to study how humans process visual information. In medical imaging, eye-tracking has become a popular technique in many applications to reveal how visual search and recognition tasks are performed, providing information that can improve human performance. In this paper, we present a comprehensive review of eye-tracking studies conducted with medical images and videos for diverse research purposes, including the identification of the degree of expertise, development of training, and understanding and modeling of visual search patterns. In addition, we present our recent eye-tracking study that involves a large number of screening mammograms viewed by experienced breast radiologists. Based on the eye-tracking data, we evaluate the plausibility of predicting visual attention by computational models.

Eye tracking has been used by many researchers to try to shed light on the perceptual processes involved in medical image perception. Despite a large volume of data having been published regarding radiologist viewing patterns for static images, and more recently for stacked imaging modalities, little has been produced concerning angiographic images, which commonly have substantially different characteristics. A study was performed in which 7 expert radiologists viewed a range of digital subtraction angiograms of the peripheral vascular system. Initial results are presented. The observers were free to control the rate at which they viewed the images. Eye position data was recorded for each participant using Tobii TX300 eyetrackers. Analysis was performed in Tobii Studio software and included qualitative analysis of gaze pattern and analysis of metrics including first and total fixation duration etc. for areas of clinical interest. Early results indicate that experts briefly fixate on lesions but do not dwell in the area, rather continuing to inspect the more distal vascular segments before returning. Some individual variation was noted. Further research is required and ongoing.

Direct volume rendering (DVR) visualization helps interpretation because it allows users to focus attention on the subset of volumetric data that is of most interest to them. The ideal visualization of the features of interest (FOIs) in a volume, however, is still a major challenge. The clear depiction of FOIs depends on accurate identification of the FOIs and appropriate specification of the optical parameters via transfer function (TF) design and it is typically a repetitive trial‐and‐error process. We address this challenge by introducing a new method that uses contextual saliency information to group the voxels along a viewing ray into distinct FOIs where ‘contextual saliency’ is a biologically inspired attribute that aids the identification of features that the human visual system considers important. The saliency information is also used to automatically define the optical parameters that emphasize the visual depiction of the FOIs in DVR. We demonstrate the capabilities of our method by its application to a variety of volumetric data sets and highlight its advantages by comparison to current state‐of‐the‐art ray profile analysis methods.