A Holistic Multimedia System for Gastrointestinal Tract Disease Detection

Analysis of medical videos for detection of abnormalities and diseases requires both high precision and recall, but also real-time processing for live feedback and scalability for massive screening of entire populations. Existing work on this field does not provide the necessary combination of retrieval accuracy and performance.; AB@In this paper, a multimedia system is presented where the aim is to tackle automatic analysis of videos from the human gastrointestinal (GI) tract. The system includes the whole pipeline from data collection, processing and analysis, to visualization. The system combines filters using machine learning, image recognition and extraction of global and local image features. Furthermore, it is built in a modular way so that it can easily be extended. At the same time, it is developed for efficient processing in order to provide real-time feedback to the doctors. Our experimental evaluation proves that our system has detection and localisation accuracy at least as good as existing systems for polyp detection, it is capable of detecting a wider range of diseases, it can analyze video in real-time, and it has a low resource consumption for scalability.

[1]  Max Q.-H. Meng,et al.  Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments , 2009, Comput. Biol. Medicine.

[2]  Michael Riegler,et al.  ClusterTag: Interactive Visualization, Clustering and Tagging Tool for Big Image Collections , 2017, ICMR.

[3]  Jingkuan Song.  Effective hashing for large-scale multimedia search , 2013, SIGMOD'13 PhD Symposium.

[4]  Klaus Schöffmann,et al.  Detection of circular content area in endoscopic videos , 2013, Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems.

[5]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[6]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[7]  Jung-Hwan Oh,et al.  Polyp Detection in Colonoscopy Video using Elliptical Shape Feature , 2007, 2007 IEEE International Conference on Image Processing.

[8]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[9]  Klaus Schöffmann,et al.  Relevance Segmentation of Laparoscopic Videos , 2013, 2013 IEEE International Symposium on Multimedia.

[10]  Michael Riegler,et al.  KVASIR: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection , 2017, MMSys.

[11]  A. Uhl,et al.  Computer-Aided Decision Support Systems for Endoscopy in the Gastrointestinal Tract: A Review , 2011, IEEE Reviews in Biomedical Engineering.

[12]  J Van Dam,et al.  Advances in diagnostic and therapeutic endoscopy. , 2000, The Medical clinics of North America.

[13]  Martha Larson,et al.  How 'How' Reflects What's What: Content-based Exploitation of How Users Frame Social Images , 2014, ACM Multimedia.

[14]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Bill Buckles,et al.  Bleeding detection from capsule endoscopy videos , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Marcin Polkowski,et al.  Quality indicators for colonoscopy and the risk of interval cancer. , 2010, The New England journal of medicine.

[17]  Atle Fretheim,et al.  Flexible sigmoidoscopy versus faecal occult blood testing for colorectal cancer screening in asymptomatic individuals. , 2013, The Cochrane database of systematic reviews.

[18]  Mingda Zhou,et al.  Polyp detection and radius measurement in small intestine using video capsule endoscopy , 2014, 2014 7th International Conference on Biomedical Engineering and Informatics.

[19]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[20]  Max Q.-H. Meng,et al.  Tumor Recognition in Wireless Capsule Endoscopy Images Using Textural Features and SVM-Based Feature Selection , 2012, IEEE Transactions on Information Technology in Biomedicine.

[21]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Michael Riegler,et al.  Expert driven semi-supervised elucidation tool for medical endoscopic videos , 2015, MMSys.

[23]  Ilangko Balasingham,et al.  A microwave imaging-based 3D localization algorithm for an in-body RF source as in wireless capsule endoscopes , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[24]  Ilangko Balasingham,et al.  Wireless communication link for capsule endoscope at 600 MHz , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[25]  Michael Riegler,et al.  Nerthus: A Bowel Preparation Quality Video Dataset , 2017, MMSys.

[26]  Klaus Schöffmann,et al.  Improving encoding efficiency of endoscopic videos by using circle detection based border overlays , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[27]  Jung-Hwan Oh,et al.  Abnormal image detection in endoscopy videos using a filter bank and local binary patterns , 2014, Neurocomputing.

[28]  Mathias Lux.  LIRE: open source image retrieval in Java , 2013, MM '13.

[29]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[30]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[31]  Lars Aabakken,et al.  Image documentation of endoscopic findings in ulcerative colitis: photographs or video clips? , 2005, Gastrointestinal endoscopy.

[32]  Carsten Griwodz,et al.  Bagadus: an integrated system for arena sports analytics: a soccer case study , 2013, MMSys.

[33]  C Senore,et al.  European guidelines for quality assurance in colorectal cancer screening and diagnosis. First Edition – Organisation , 2012, Endoscopy.

[34]  Jung-Hwan Oh,et al.  Detection of Quality Visualization of Appendiceal Orifices Using Local Edge Cross-Section Profile Features and Near Pause Detection , 2010, IEEE Transactions on Biomedical Engineering.

[35]  Xiaoyi Jiang,et al.  Colorectal Polyps Detection Using Texture Features and Support Vector Machine , 2008, MDA.

[36]  R. Tibshirani,et al.  Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[37]  Shinji Tanaka,et al.  Endoscopic prediction of deep submucosal invasive carcinoma: validation of the narrow-band imaging international colorectal endoscopic (NICE) classification. , 2013, Gastrointestinal endoscopy.

[38]  Jung-Hwan Oh,et al.  Polyp-Alert: Near real-time feedback during colonoscopy , 2015, Comput. Methods Programs Biomed..

[39]  Nima Tajbakhsh,et al.  Automated Polyp Detection in Colonoscopy Videos Using Shape and Context Information , 2016, IEEE Transactions on Medical Imaging.

[40]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[41]  Luís A. Alexandre,et al.  Polyp Detection in Endoscopic Video Using SVMs , 2007, PKDD.

[42]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[43]  N Segnan,et al.  European guidelines for quality assurance in colorectal cancer screening and diagnosis. First Edition – Executive summary , 2012, Endoscopy.

[44]  Isabel N. Figueiredo,et al.  Automated Polyp Detection in Colon Capsule Endoscopy , 2013, IEEE Transactions on Medical Imaging.

[45]  Michael Riegler,et al.  From Annotation to Computer-Aided Diagnosis , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[46]  Michael Riegler,et al.  GPU-Accelerated Real-Time Gastrointestinal Diseases Detection , 2016, 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS).

[47]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[48]  Maya Gokhale,et al.  Accelerating a Random Forest Classifier: Multi-Core, GP-GPU, or FPGA? , 2012, 2012 IEEE 20th International Symposium on Field-Programmable Custom Computing Machines.

[49]  Michael Riegler,et al.  EIR — Efficient computer aided diagnosis framework for gastrointestinal endoscopies , 2016, 2016 14th International Workshop on Content-Based Multimedia Indexing (CBMI).

[50]  Jung-Hwan Oh,et al.  Part-Based Multiderivative Edge Cross-Sectional Profiles for Polyp Detection in Colonoscopy , 2014, IEEE Journal of Biomedical and Health Informatics.

[51]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[52]  Christine Chin,et al.  Learning in Science: A Comparison of Deep and Surface Approaches. , 2000 .

[53]  R. Doraiswami,et al.  Real-time image processing system for endoscopic applications , 2003, CCECE 2003 - Canadian Conference on Electrical and Computer Engineering. Toward a Caring and Humane Technology (Cat. No.03CH37436).

[54]  Mathias Lux,et al.  Visual information retrieval using Java and LIRE , 2013, SIGIR '12.

[55]  Moncef Gabbouj,et al.  Feature selection for content-based image retrieval , 2008, Signal Image Video Process..

[56]  Yi Wang,et al.  Computer-aided detection of retroflexion in colonoscopy , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[57]  Jung-Hwan Oh,et al.  Near Real-Time Retroflexion Detection in Colonoscopy , 2013, IEEE Journal of Biomedical and Health Informatics.

[58]  Junliang Yang,et al.  DEM analysis of soil fabric effects on behaviour of sand S. YIMSIRI and K. SOGA (2010). Géotechnique 60, No. 6, 483–495 , 2011 .

[59]  N. Segnan,et al.  European guidelines for quality assurance in colorectal cancer screening and diagnosis. First Edition – Principles of evidence assessment and methods for reaching recommendations , 2012, Endoscopy.

[60]  Ramón López de Mántaras,et al.  A distance-based attribute selection measure for decision tree induction , 1991, Machine Learning.

[61]  Carsten Griwodz,et al.  Bagadus: An integrated real-time system for soccer analytics , 2014, ACM Trans. Multim. Comput. Commun. Appl..

[62]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[63]  Fernando Vilariño,et al.  Texture-Based Polyp Detection in Colonoscopy , 2009, Bildverarbeitung für die Medizin.