Detecting and Locating Gastrointestinal Anomalies Using Deep Learning and Iterative Cluster Unification

This paper proposes a novel methodology for automatic detection and localization of gastrointestinal (GI) anomalies in endoscopic video frame sequences. Training is performed with weakly annotated images, using only image-level, semantic labels instead of detailed, and pixel-level annotations. This makes it a cost-effective approach for the analysis of large videoendoscopy repositories. Other advantages of the proposed methodology include its capability to suggest possible locations of GI anomalies within the video frames, and its generality, in the sense that abnormal frame detection is based on automatically derived image features. It is implemented in three phases: 1) it classifies the video frames into abnormal or normal using a weakly supervised convolutional neural network (WCNN) architecture; 2) detects salient points from deeper WCNN layers, using a deep saliency detection algorithm; and 3) localizes GI anomalies using an iterative cluster unification (ICU) algorithm. ICU is based on a pointwise cross-feature-map (PCFM) descriptor extracted locally from the detected salient points using information derived from the WCNN. Results, from extensive experimentation using publicly available collections of gastrointestinal endoscopy video frames, are presented. The data sets used include a variety of GI anomalies. Both anomaly detection and localization performance achieved, in terms of the area under receiver operating characteristic (AUC), were >80%. The highest AUC for anomaly detection was obtained on conventional gastroscopy images, reaching 96%, and the highest AUC for anomaly localization was obtained on wireless capsule endoscopy images, reaching 88%.

[1]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[2]  Marios Anthimopoulos,et al.  Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[3]  Ting Rui,et al.  Convolutional neural network feature maps selection based on LDA , 2017, Multimedia Tools and Applications.

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Andreas Uhl,et al.  CNN transfer learning for the automated diagnosis of celiac disease , 2016, 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[6]  Frans Coenen,et al.  Visual attribute classification using feature selection and convolutional neural network , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[7]  Demissie Alemayehu,et al.  Applications of ROC analysis in medical research: recent developments and future directions. , 2012, Academic radiology.

[8]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[9]  Max Q.-H. Meng,et al.  A study on automated segmentation of blood regions in Wireless Capsule Endoscopy images using fully convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[10]  Dimitris A. Karras,et al.  Computer-aided tumor detection in endoscopic video using color wavelet features , 2003, IEEE Transactions on Information Technology in Biomedicine.

[11]  Krish Ragunath,et al.  Optimising the performance and interpretation of small bowel capsule endoscopy , 2017, Frontline Gastroenterology.

[12]  Dimitris K. Iakovidis,et al.  Automatic lesion detection in capsule endoscopy based on color saliency: closer to an essential adjunct for reviewing software. , 2014, Gastrointestinal endoscopy.

[13]  Jordi Vitrià,et al.  Generic Feature Learning for Wireless Capsule Endoscopy Analysis , 2016, Comput. Biol. Medicine.

[14]  Yiqun Jia.  Polyps auto-detection in Wireless Capsule Endoscopy images using improved method based on image segmentation , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[15]  George Papandreou,et al.  Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Nikolaos G. Bourbakis,et al.  Detection of Small Bowel Polyps and Ulcers in Wireless Capsule Endoscopy Videos , 2011, IEEE Transactions on Biomedical Engineering.

[17]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[18]  Yoshua Bengio,et al.  Object Recognition with Gradient-Based Learning , 1999, Shape, Contour and Grouping in Computer Vision.

[19]  Gregory G. Slabaugh,et al.  Automatic Segmentation of Polyps in Colonoscopic Narrow-Band Imaging Data , 2012, IEEE Transactions on Biomedical Engineering.

[20]  Max Q.-H. Meng,et al.  Bleeding Frame and Region Detection in the Wireless Capsule Endoscopy Video , 2016, IEEE Journal of Biomedical and Health Informatics.

[21]  Aymeric Histace,et al.  Towards Real-Time Polyp Detection in Colonoscopy Videos: Adapting Still Frame-Based Methodologies for Video Sequences Analysis , 2017, CARE/CLIP@MICCAI.

[22]  D. Iakovidis,et al.  Software for enhanced video capsule endoscopy: challenges for essential progress , 2015, Nature Reviews Gastroenterology &Hepatology.

[23]  Carsten Rother,et al.  Learning discriminative localization from weakly labeled data , 2014, Pattern Recognit..

[24]  Dimitris K. Iakovidis,et al.  KID Project: an internet-based digital video atlas of capsule endoscopy for research purposes , 2017, Endoscopy International Open.

[25]  Fons van der Sommen,et al.  Supportive automatic annotation of early esophageal cancer using local gabor and color features , 2014, Neurocomputing.

[26]  Nassir Navab,et al.  Automatic Quantification of Tumour Hypoxia From Multi-Modal Microscopy Images Using Weakly-Supervised Learning Methods , 2017, IEEE Transactions on Medical Imaging.

[27]  Max Q.-H. Meng,et al.  Improved Bag of Feature for Automatic Polyp Detection in Wireless Capsule Endoscopy Images , 2016, IEEE Transactions on Automation Science and Engineering.

[28]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[29]  Carmen C. Y. Poon,et al.  Automatic Detection and Classification of Colorectal Polyps by Transferring Low-Level CNN Features From Nonmedical Domain , 2017, IEEE Journal of Biomedical and Health Informatics.

[30]  Jiebo Luo,et al.  Deep sparse feature selection for computer aided endoscopy diagnosis , 2015, Pattern Recognit..

[31]  A. Uhl,et al.  Computer-Aided Decision Support Systems for Endoscopy in the Gastrointestinal Tract: A Review , 2011, IEEE Reviews in Biomedical Engineering.

[32]  Huaici Zhao,et al.  Computer-Aided Endoscopic Diagnosis Without Human-Specific Labeling , 2016, IEEE Transactions on Biomedical Engineering.

[33]  Eric Goldberg,et al.  Detection of Lesions During Capsule Endoscopy: Physician Performance Is Disappointing , 2012, The American Journal of Gastroenterology.

[34]  S. T. Devarakonda,et al.  A convolutional neural network approach for abnormality detection in Wireless Capsule Endoscopy , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[35]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[36]  Isabel N. Figueiredo,et al.  Automated Polyp Detection in Colon Capsule Endoscopy , 2013, IEEE Transactions on Medical Imaging.

[37]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[38]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[39]  Rong Zhang,et al.  Lesion detection of endoscopy images based on convolutional neural network features , 2015, 2015 8th International Congress on Image and Signal Processing (CISP).

[40]  Andreas Uhl,et al.  Colonic Polyp Classification with Convolutional Neural Networks , 2016, 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS).

[41]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[42]  Xiao Wu,et al.  Automatic content understanding with cascaded spatial-temporal deep framework for capsule endoscopy videos , 2017, Neurocomputing.

[43]  David Saad,et al.  On-Line Learning in Neural Networks , 1999 .

[44]  Youngbae Hwang,et al.  Application of Artificial Intelligence in Capsule Endoscopy: Where Are We Now? , 2018, Clinical endoscopy.

[45]  Tom Fawcett,et al.  Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions , 1997, KDD.

[46]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[47]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[48]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[49]  Bram van Ginneken,et al.  Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images , 2016, IEEE Transactions on Medical Imaging.

[50]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[51]  Shuai Wang,et al.  UDSFS: Unsupervised deep sparse feature selection , 2016, Neurocomputing.

[52]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[53]  Gustavo K. Rohde,et al.  Epithelium-Stroma Classification via Convolutional Neural Networks and Unsupervised Domain Adaptation in Histopathological Images , 2017, IEEE Journal of Biomedical and Health Informatics.

[54]  Ivan Laptev,et al.  Is object localization for free? - Weakly-supervised learning with convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Aymeric Histace,et al.  Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results From the MICCAI 2015 Endoscopic Vision Challenge , 2017, IEEE Transactions on Medical Imaging.

[56]  Huaici Zhao,et al.  Computer aided endoscope diagnosis via weakly labeled data mining , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[57]  Dimitris K. Iakovidis,et al.  Weakly-supervised Convolutional learning for detection of inflammatory gastrointestinal lesions , 2016, 2016 IEEE International Conference on Imaging Systems and Techniques (IST).

[58]  Anastasios Koulaouzidis,et al.  Automatic lesion detection in wireless capsule endoscopy — A simple solution for a complex problem , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[59]  Dimitris K. Tasoulis,et al.  Random direction divisive clustering , 2013, Pattern Recognit. Lett..

[60]  Heike Freud,et al.  On Line Learning In Neural Networks , 2016 .

[61]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[62]  Alexandros Karargyris,et al.  Optimizing lesion detection in small-bowel capsule endoscopy: from present problems to future solutions , 2015, Expert review of gastroenterology & hepatology.

[63]  Daoqiang Zhang,et al.  Topological correction of infant white matter surfaces using anatomically constrained convolutional neural network , 2019, NeuroImage.

[64]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[65]  Ronald M. Summers,et al.  Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation , 2015, IEEE Transactions on Medical Imaging.

[66]  Evaggelos Spyrou,et al.  Weakly-Supervised Lesion Detection in Video Capsule Endoscopy Based on a Bag-of-Colour Features Model , 2016, CARE@MICCAI.

[67]  Nima Tajbakhsh,et al.  Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).