Evaluation of Gaze Tracking Calibration for Longitudinal Biomedical Imaging Studies

Gaze tracking is a promising technology for studying the visual perception of clinicians during image-based medical exams. It could be used in longitudinal studies to analyze their perceptive process, explore human-machine interactions, and develop innovative computer-aided imaging systems. However, using a remote eye tracker in an unconstrained environment and over time periods of weeks requires a certain guarantee of performance to ensure that collected gaze data are fit for purpose. We report the results of evaluating eye tracking calibration for longitudinal studies. First, we tested the performance of an eye tracker on a cohort of 13 users over a period of one month. For each participant, the eye tracker was calibrated during the first session. The participants were asked to sit in front of a monitor equipped with the eye tracker, but their position was not constrained. Second, we tested the performance of the eye tracker on sonographers positioned in front of a cart-based ultrasound scanner. Experimental results show a decrease of accuracy between calibration and later testing of 0.30° and a further degradation over time at a rate of 0.13°. month−1. The overall median accuracy was 1.00° (50.9 pixels) and the overall median precision was 0.16° (8.3 pixels). The results from the ultrasonography setting show a decrease of accuracy of 0.16° between calibration and later testing. This slow degradation of gaze tracking accuracy could impact the data quality in long-term studies. Therefore, the results we present here can help in planning such long-term gaze tracking studies.

[1]  Elizabeth A Krupinski,et al.  Current perspectives in medical image perception , 2010, Attention, perception & psychophysics.

[2]  Yifan Peng,et al.  Studying Relationships between Human Gaze, Description, and Computer Vision , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Necip Berme,et al.  Eye motion parameters correlate with level of experience in video-assisted surgery: objective testing of three tasks. , 2005, Journal of laparoendoscopic & advanced surgical techniques. Part A.

[4]  Peter Corcoran,et al.  A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms , 2017, IEEE Access.

[5]  František Zahálka,et al.  Original Research , 2007 .

[6]  Ioannis Patras,et al.  Action recognition using saliency learned from recorded human gaze , 2016, Image Vis. Comput..

[7]  Shuo Wang,et al.  Predicting human gaze beyond pixels. , 2014, Journal of vision.

[8]  Guang-Zhong Yang,et al.  Extraction of visual features with eye tracking for saliency driven 2D/3D registration , 2005, Image Vis. Comput..

[9]  Fred W. Mast,et al.  What Was I Thinking? Eye-Tracking Experiments Underscore the Bias that Architecture Exerts on Nuclear Grading in Prostate Cancer , 2012, PloS one.

[10]  Claudia Mello-Thoms,et al.  Using gaze-tracking data and mixture distribution analysis to support a holistic model for the detection of cancers on mammograms. , 2008, Academic radiology.

[11]  John Paulin Hansen,et al.  Low cost vs. high-end eye tracking for usability testing , 2011, CHI Extended Abstracts.

[12]  H. A. Mooij,et al.  Visualising scanning patterns of pathologists in the grading of cervical intraepithelial neoplasia , 2003, Journal of clinical pathology.

[13]  Lindsey Cooper,et al.  The assessment of stroke multidimensional CT and MR imaging using eye movement analysis: does modality preference enhance observer performance? , 2010, Medical Imaging.

[14]  T. Crawford,et al.  How do radiologists do it? The influence of experience and training on searching for chest nodules. , 2006 .

[15]  Mark R. Wilson,et al.  Psychomotor control in a virtual laparoscopic surgery training environment: gaze control parameters differentiate novices from experts , 2010, Surgical Endoscopy.

[16]  Raymond Bertram,et al.  The Effect of Expertise on Eye Movement Behaviour in Medical Image Perception , 2013, PloS one.

[17]  Robert Rohling,et al.  An application of eyegaze tracking for designing radiologists' workstations: Insights for comparative visual search tasks , 2006, TAP.

[18]  Jayanthi Sivaswamy,et al.  Assessment of computational visual attention models on medical images , 2012, ICVGIP '12.

[19]  A. Darzi,et al.  Visual search behaviour in skeletal radiographs: a cross-specialty study. , 2007, Clinical radiology.

[20]  H L Kundel,et al.  Nature of expertise in searching mammograms for breast masses , 1996, Medical Imaging.

[21]  Linden J. Ball,et al.  Eye tracking in HCI and usability research. , 2006 .

[22]  Trafton Drew,et al.  Informatics in radiology: what can you see in a single glance and how might this guide visual search in medical images? , 2013, Radiographics : a review publication of the Radiological Society of North America, Inc.

[23]  J. Alison Noble,et al.  Fetal Ultrasound Image Classification Using a Bag-of-words Model Trained on Sonographers' Eye Movements , 2016, MIUA.

[24]  Elizabeth A Krupinski,et al.  Visual search of mammographic images: influence of lesion subtlety. , 2005, Academic radiology.

[25]  Frank Keller,et al.  Training Object Class Detectors from Eye Tracking Data , 2014, ECCV.

[26]  Beverly E. Faulkner-Jones,et al.  Eye-Tracking in the Study of Visual Expertise: Methodology and Approaches in Medicine , 2017 .

[27]  Damien Litchfield,et al.  Looking for Cancer: Expertise Related Differences in Searching and Decision Making , 2013 .

[28]  A. Venjakob Visual search, perception and cognition when reading stack mode cranial CT , 2015 .

[29]  Qi Zhao,et al.  Learning saliency-based visual attention: A review , 2013, Signal Process..

[30]  Zheru Chi,et al.  Eye tracking data guided feature selection for image classification , 2017, Pattern Recognit..

[31]  E. Conant,et al.  Holistic component of image perception in mammogram interpretation: gaze-tracking study. , 2007, Radiology.

[32]  R. Hanajima,et al.  Where Do Neurologists Look When Viewing Brain CT Images? An Eye-Tracking Study Involving Stroke Cases , 2011, PloS one.

[33]  Peter D. Lawrence,et al.  Improving the Accuracy and Reliability of Remote System-Calibration-Free Eye-Gaze Tracking , 2009, IEEE Transactions on Biomedical Engineering.

[34]  Yi Yang,et al.  Weakly Supervised Human Fixations Prediction , 2016, IEEE Transactions on Cybernetics.

[35]  E. Krupinski,et al.  Characterizing the development of visual search expertise in pathology residents viewing whole slide images. , 2013, Human pathology.

[36]  Yuan Gao,et al.  Detection and Characterization of the Fetal Heartbeat in Free-hand Ultrasound Sweeps with Weakly-supervised Two-streams Convolutional Networks , 2017, MICCAI.

[37]  Theo Gevers,et al.  Accurate Eye Center Location through Invariant Isocentric Patterns , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  H L Kundel,et al.  The influence of prior knowledge on visual search strategies during the viewing of chest radiographs. , 1969, Radiology.

[39]  Zheru Chi,et al.  Relative Saliency Model over Multiple Images with an Application to Yarn Surface Evaluation , 2014, IEEE Transactions on Cybernetics.

[40]  Karla K. Evans,et al.  If You Don’t Find It Often, You Often Don’t Find It: Why Some Cancers Are Missed in Breast Cancer Screening , 2013, PloS one.

[41]  Christine Cavaro-Ménard,et al.  Eye-position recording during brain MRI examination to identify and characterize steps of glioma diagnosis , 2010, Medical Imaging.

[42]  E. Lansdown,et al.  VISUAL SEARCH PATTERNS OF RADIOLOGISTS IN TRAINING. , 1963, Radiology.

[43]  E. Krupinski,et al.  Eye-movement study and human performance using telepathology virtual slides: implications for medical education and differences with experience. , 2006, Human pathology.

[44]  A. Kosevoi-Tichie,et al.  THU0583 Does Eye Gaze Tracking Have the Ability to Assess How Rheumatologists Evaluate Musculoskeletal Ultrasound Images? , 2015 .

[45]  Edwige Pissaloux,et al.  Towards activity recognition from eye-movements using contextual temporal learning , 2017, Integr. Comput. Aided Eng..

[46]  Nadine Sarter,et al.  The Effects of Data Density, Display Organization, and Stress on Search Performance: An Eye Tracking Study of Clutter , 2017, IEEE Transactions on Human-Machine Systems.

[47]  Tad T. Brunyé,et al.  Eye Movements as an Index of Pathologist Visual Expertise: A Pilot Study , 2014, PloS one.

[48]  Jayanthi Sivaswamy,et al.  Visual saliency based bright lesion detection and discrimination in retinal images , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[49]  George W. Quinn,et al.  IREX VI : Temporal Stability of Iris Recognition Accuracy , 2013 .

[50]  Elizabeth A. Krupinski,et al.  Understanding Visual Search Patterns of Dermatologists Assessing Pigmented Skin Lesions Before and After Online Training , 2014, Journal of Digital Imaging.

[51]  Sung Wook Baik,et al.  Prioritization of brain MRI volumes using medical image perception model and tumor region segmentation , 2013, Comput. Biol. Medicine.

[52]  Emma Helbren,et al.  Tracking eye gaze during interpretation of endoluminal three-dimensional CT colonography: visual perception of experienced and inexperienced readers. , 2014, Radiology.

[53]  T Kyle Harrison,et al.  Preliminary Experience Using Eye‐Tracking Technology to Differentiate Novice and Expert Image Interpretation for Ultrasound‐Guided Regional Anesthesia , 2018, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[54]  Gezheng Wen,et al.  Computational assessment of visual search strategies in volumetric medical images , 2016, Journal of medical imaging.

[55]  Cristian Sminchisescu,et al.  Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Mark R. Wilson,et al.  Visual expertise in detecting and diagnosing skeletal fractures , 2013, Skeletal Radiology.

[57]  Mark F McEntee,et al.  The effect of abnormality-prevalence expectation on expert observer performance and visual search. , 2011, Radiology.

[58]  Gabor Fichtinger,et al.  An objective and automated method for assessing surgical skill in endoscopic sinus surgery using eye‐tracking and tool‐motion data , 2012, International forum of allergy & rhinology.

[59]  Patrick Le Callet,et al.  Visual Attention and Applications in Multimedia Technologies , 2013, Proceedings of the IEEE.

[60]  Claudia Mello-Thoms,et al.  How does the perception of a lesion influence visual search strategy in mammogram reading? , 2006, Academic radiology.

[61]  Kenneth Holmqvist,et al.  Visual expertise in paediatric neurology. , 2012, European journal of paediatric neurology : EJPN : official journal of the European Paediatric Neurology Society.

[62]  Eric T. Greenlee,et al.  Which Eye Tracker Is Right for Your Research? Performance Evaluation of Several Cost Variant Eye Trackers , 2016 .

[63]  Elena Gaudioso,et al.  Evaluation of temporal stability of eye tracking algorithms using webcams , 2016, Expert Syst. Appl..

[64]  Robert J. K. Jacob,et al.  Eye tracking in human-computer interaction and usability research : Ready to deliver the promises , 2002 .

[65]  J. Alison Noble,et al.  SonoEyeNet: Standardized fetal ultrasound plane detection informed by eye tracking , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).