Visual Perception Based Assistance for Fundus Image

Diabetic Retinopathy (DR) is a condition where individuals with diabetes develop a disease in the inner wall of eye known as retina. DR is a major cause of visual impairments and early detection can prevent vision loss. Use of automatic systems for DR diagnosis is limited due to their lower accuracy. As an alternative, reading-centers are becoming popular in real-world scenarios. Reading center is a facility where retinal images coming from various sources are stored and trained personals(who might not be experts) analyze them. In this thesis we look at techniques to increase efficiency of DR image-readers working in reading centers. The first half of this thesis aims at identifying efficient image-reading technique which is both fast and accurate. Towards this end we have conducted an eye-tracking study with medical experts while they were reading images for DR diagnosis. The analysis shows that experts employ mainly two types of reading strategies: dwelling and tracing. Dwelling strategy appears to be accurate and faster than tracing strategy. Eye movements of all the experts are combined in a novel way to extract an optimal image scanning strategy, which can be recommended to image-readers for efficient diagnosis. In order to increase the efficiency further, we propose a technique where saliency of lesions can be boosted for better visibility of lesions. This is named as an Assistive Lesion Emphasis System(ALES) and demonstrated in the second half of the thesis. ALES is developed as a two stage system: saliency detection and lesion emphasis. Two biologically inspired saliency models, which mimic human visual system, are designed using unsupervised and supervised techniques. Unsupervised saliency model is inspired from human visual system and achieved 10% higher recall than other existing saliency models when compared with average gazemap of 15 retinal experts. Supervised saliency model developed as deep learning based implementation of biologically inspired saliency model proposed by IttiKoch(Itti, L., Koch, C. and Niebur, E., 1998) and achieves 10% to 20% higher AUC compared

[1]  Ja Wilson,et al.  Principles and practice of screening for disease , 1968 .

[2]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[3]  D. G. Albrecht,et al.  Striate cortex of monkey and cat: contrast response function. , 1982, Journal of neurophysiology.

[4]  E. Krupinski,et al.  Searching for lung nodules. Visual dwell indicates locations of false-positive and false-negative decisions. , 1989, Investigative radiology.

[5]  Susan L. Franzel,et al.  Guided search: an alternative to the feature integration model for visual search. , 1989, Journal of experimental psychology. Human perception and performance.

[6]  B. Hoogwerf,et al.  Association of elevated serum lipid levels with retinal hard exudate in diabetic retinopathy. Early Treatment Diabetic Retinopathy Study (ETDRS) Report 22. , 1996, Archives of ophthalmology.

[7]  C. Sinthanayothin,et al.  Automated localisation of the optic disc, fovea, and retinal blood vessels from digital colour fundus images , 1999, The British journal of ophthalmology.

[8]  K. Berbaum,et al.  Gaze dwell times on acute trauma injuries missed because of satisfaction of search. , 2001, Academic radiology.

[9]  Yiming Wang,et al.  Illumination normalization of retinal images using sampling and interpolation , 2001, SPIE Medical Imaging.

[10]  Aidong Zhang,et al.  Semantics-Based Image Retrieval by Region Saliency , 2002, CIVR.

[11]  Jang-Kyoo Shin,et al.  Biologically Inspired Saliency Map Model for Bottom-up Visual Attention , 2002, Biologically Motivated Computer Vision.

[12]  S. Hori Guidelines for the Management of Diabetic Retinopathy for the Internist , 2002 .

[13]  C. Mello-Thoms Perception of breast cancer: eye-position analysis of mammogram interpretation. , 2003, Academic radiology.

[14]  D. Squirrell,et al.  Screening for Diabetic Retinopathy , 2003, Journal of the Royal Society of Medicine.

[15]  Nuno Vasconcelos,et al.  Discriminant Saliency for Visual Recognition from Cluttered Scenes , 2004, NIPS.

[16]  Frédo Durand,et al.  De-emphasis of distracting image regions using texture power maps , 2005, APGV '05.

[17]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[18]  Enrico Grisan,et al.  Luminosity and contrast normalization in retinal images , 2005, Medical Image Anal..

[19]  Simone Frintrop,et al.  Goal-Directed Search with a Top-Down Modulated Computational Attention System , 2005, DAGM-Symposium.

[20]  David Cornforth,et al.  Colour normalisation to reduce inter-patient and intra-patient variability in microaneurysm detection in colour retinal images , 2005 .

[21]  Enrico Grisan,et al.  Model-based illumination correction in retinal images , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[22]  Matthew de Brecht,et al.  A neural network implementation of a saliency map model , 2006, Neural Networks.

[23]  D. Hansell,et al.  Thin-section CT of the lungs: eye-tracking analysis of the visual approach to reading tiled and stacked display formats. , 2006, European journal of radiology.

[24]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[25]  Amitabh Varshney,et al.  Saliency-guided Enhancement for Volume Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[26]  E. Krupinski,et al.  Eye-movement study and human performance using telepathology virtual slides: implications for medical education and differences with experience. , 2006, Human pathology.

[27]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[28]  Joni-Kristian Kämäräinen,et al.  The DIARETDB1 Diabetic Retinopathy Database and Evaluation Protocol , 2007, BMVC.

[29]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  E. Conant,et al.  Holistic component of image perception in mammogram interpretation: gaze-tracking study. , 2007, Radiology.

[31]  Heikki Kälviäinen,et al.  DIARETDB 0 : Evaluation Database and Methodology for Diabetic Retinopathy Algorithms , 2007 .

[32]  Wei Jin,et al.  Enhancing retinal image by the Contourlet transform , 2007, Pattern Recognit. Lett..

[33]  Asoke K. Nandi,et al.  Novel and adaptive contribution of the red channel in pre-processing of colour fundus images , 2007, J. Frankl. Inst..

[34]  Liqing Zhang,et al.  Dynamic visual attention: searching for coding length increments , 2008, NIPS.

[35]  Matei Mancas,et al.  Image perception : Relative influence of bottom-up and top-down attention , 2008 .

[36]  Jayanthi Sivaswamy,et al.  Colour Retinal Image Enhancement Based on Domain Knowledge , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[37]  Dwarikanath Mahapatra,et al.  Registration of dynamic renal MR images using neurobiological model of saliency , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[38]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[39]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[40]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[41]  M. Wintermark,et al.  Acute stroke triage to intravenous thrombolysis and other therapies with advanced CT or MR imaging: pro CT. , 2009, Radiology.

[42]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[44]  R. Johnston,et al.  A Study of Radiologists Viewing Multiple CT Scans Using An Eyetracking Device TR 89-039 December , 1989 , 2010 .

[45]  Eberhart Zrenner,et al.  A domain model of a clinical reading center - Design and implementation , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[46]  Marcus Nyström,et al.  An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data , 2010, Behavior research methods.

[47]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[48]  Kok-Lim Low,et al.  Saliency retargeting: An approach to enhance image aesthetics , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[49]  R. Hanajima,et al.  Where Do Neurologists Look When Viewing Brain CT Images? An Eye-Tracking Study Involving Stroke Cases , 2011, PloS one.

[50]  Ching-Te Chiu,et al.  BiTA/SWCE: Image Enhancement With Bilateral Tone Adjustment and Saliency Weighted Contrast Enhancement , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[51]  A. Sugimoto,et al.  Saliency-based image editing for guiding visual attention , 2011, PETMEI '11.

[52]  Christof Koch,et al.  Learning a saliency map using fixated locations in natural scenes. , 2011, Journal of vision.

[53]  A. Arieno,et al.  CAD May Not be Necessary for Microcalcifications in the Digital era, CAD May Benefit Radiologists for Masses , 2012, Journal of clinical imaging science.

[54]  Jayanthi Sivaswamy,et al.  Assessment of computational visual attention models on medical images , 2012, ICVGIP '12.

[55]  Kenneth W. Tobin,et al.  Exudate-based diabetic macular edema detection in fundus images using publicly available datasets , 2012, Medical Image Anal..

[56]  Oleg S. Pianykh,et al.  Digital Imaging and Communications in Medicine (DICOM) , 2017, Radiopaedia.org.

[57]  E. Krupinski,et al.  Characterizing the development of visual search expertise in pathology residents viewing whole slide images. , 2013, Human pathology.

[58]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Sven Loncaric,et al.  Diabetic retinopathy image database(DRiDB): A new database for diabetic retinopathy screening programs research , 2013, 2013 8th International Symposium on Image and Signal Processing and Analysis (ISPA).

[60]  T. Peto,et al.  Crowdsourcing as a Novel Technique for Retinal Fundus Photography Classification: Analysis of Images in the EPIC Norfolk Cohort on Behalf of the UKBiobank Eye and Vision Consortium , 2013, PloS one.

[61]  Ignace Hooge,et al.  Scan path entropy and arrow plots: capturing scanning behavior of multiple observers , 2013, Front. Psychol..

[62]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  J. Wolfe,et al.  The Invisible Gorilla Strikes Again , 2013, Psychological science.

[64]  Joseph M. Reinhardt,et al.  Splat Feature Classification With Application to Retinal Hemorrhage Detection in Fundus Images , 2013, IEEE Transactions on Medical Imaging.

[65]  Qi Zhao,et al.  SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[66]  Jayanthi Sivaswamy,et al.  An assistive annotation system for retinal images , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[67]  Hamid R. Tizhoosh,et al.  Medical Image Classification via SVM Using LBP Features from Saliency-Based Folded Data , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[68]  J. V. van Merriënboer,et al.  Expertise in clinical pathology: combining the visual and cognitive perspective , 2015, Advances in health sciences education : theory and practice.

[69]  George Azzopardi,et al.  Trainable COSFIRE filters for vessel delineation with application to retinal images , 2015, Medical Image Anal..

[70]  Na Li,et al.  Learning-Based Visual Saliency Model for Detecting Diabetic Macular Edema in Retinal Image , 2016, Comput. Intell. Neurosci..

[71]  B. Nayak,et al.  Prevalence of diabetic retinopathy in India: The All India Ophthalmological Society Diabetic Retinopathy Eye Screening Study 2014 , 2016, Indian journal of ophthalmology.

[72]  J. Lortet-Tieulent,et al.  Breast Cancer Screening for Women at Average Risk: 2015 Guideline Update From the American Cancer Society , 2016 .

[73]  Dwarikanath Mahapatra,et al.  Visual saliency-based active learning for prostate magnetic resonance imaging segmentation , 2016, Journal of medical imaging.

[74]  David Dagan Feng,et al.  Plane identification in fetal ultrasound images using saliency maps and convolutional neural networks , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[75]  Subhashis Banerjee,et al.  A Novel GBM Saliency Detection Model Using Multi-Channel MRI , 2016, PloS one.

[76]  R. Venkatesh Babu,et al.  DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations , 2015, IEEE Transactions on Image Processing.

[77]  Frédo Durand,et al.  What Do Different Evaluation Metrics Tell Us About Saliency Models? , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.