Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy

The detection and removal of precancerous polyps via colonoscopy is the gold standard for the prevention of colon cancer. However, the detection rate of adenomatous polyps can vary significantly among endoscopists. Here, we show that a machine-learning algorithm can detect polyps in clinical colonoscopies, in real time and with high sensitivity and specificity. We developed the deep-learning algorithm by using data from 1,290 patients, and validated it on newly collected 27,113 colonoscopy images from 1,138 patients with at least one detected polyp (per-image-sensitivity, 94.38%; per-image-specificity, 95.92%; area under the receiver operating characteristic curve, 0.984), on a public database of 612 polyp-containing images (per-image-sensitivity, 88.24%), on 138 colonoscopy videos with histologically confirmed polyps (per-image-sensitivity of 91.64%; per-polyp-sensitivity, 100%), and on 54 unaltered full-range colonoscopy videos without polyps (per-image-specificity, 95.40%). By using a multi-threaded processing system, the algorithm can process at least 25 frames per second with a latency of 76.80 ± 5.60 ms in real-time video analysis. The software may aid endoscopists while performing colonoscopies, and help assess differences in polyp and adenoma detection performance among endoscopists.A deep-learning algorithm can detect polyps in the colon in real time and with high sensitivity and specificity, according to validation studies with prospectively collected images and videos from colonoscopies performed in 1,138 patients.

[1]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[2]  D. Rex,et al.  Colonoscopic miss rates of adenomas determined by back-to-back colonoscopies. , 1997, Gastroenterology.

[3]  C. Mulrow,et al.  Colorectal cancer screening: clinical guidelines and rationale. , 1997, Gastroenterology.

[4]  L. Seeff,et al.  How many endoscopies are performed for colorectal cancer screening? Results from CDC's survey of endoscopic capacity. , 2004, Gastroenterology.

[5]  D. Heresbach,et al.  Miss rate for colorectal neoplastic polyps: a prospective multicenter study of back-to-back video colonoscopies , 2008, Endoscopy.

[6]  David Gur,et al.  Area under the Free‐Response ROC Curve (FROC) and a Related Summary Index , 2009, Biometrics.

[7]  A. Jemal,et al.  Annual report to the nation on the status of cancer, 1975‐2006, featuring colorectal cancer trends and impact of interventions (risk factors, screening, and treatment) to reduce future rates , 2010, Cancer.

[8]  J. Cha,et al.  Participation by experienced endoscopy nurses increases the detection rate of colon polyps during a screening colonoscopy: a multicenter, prospective, randomized study. , 2011, Gastrointestinal endoscopy.

[9]  M. Wallace,et al.  Trainee participation is associated with increased small adenoma detection. , 2011, Gastrointestinal endoscopy.

[10]  H. Brenner,et al.  Protection From Colorectal Cancer After Colonoscopy , 2011, Annals of Internal Medicine.

[11]  K. Chayama,et al.  Computer-aided system for predicting the histology of colorectal tumors by using narrow-band imaging magnifying colonoscopy (with video). , 2012, Gastrointestinal endoscopy.

[12]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[13]  A. Zauber,et al.  Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. , 2012, The New England journal of medicine.

[14]  D. Han,et al.  The Miss Rate for Colorectal Adenoma Determined by Quality-Adjusted, Back-to-Back Colonoscopies , 2012, Gut and liver.

[15]  Douglas K Rex,et al.  Guidelines for colonoscopy surveillance after screening and polypectomy: a consensus update by the US Multi-Society Task Force on Colorectal Cancer. , 2012, Gastroenterology.

[16]  A. M. Leufkens,et al.  Factors influencing the miss rate of polyps in a back-to-back colonoscopy study , 2012, Endoscopy.

[17]  Kenshi Yao,et al.  The endoscopic diagnosis of early gastric cancer , 2014, Annals of gastroenterology.

[18]  M. Neurath,et al.  Advanced Endoscopic Imaging , 2013, Diagnostic and therapeutic endoscopy.

[19]  A. Jemal,et al.  Cancer statistics, 2013 , 2013, CA: a cancer journal for clinicians.

[20]  Uzma D. Siddiqui,et al.  Nurse Observation During Colonoscopy Increases Polyp Detection: A Randomized Prospective Study , 2013, The American Journal of Gastroenterology.

[21]  Jung-Hwan Oh,et al.  Part-Based Multiderivative Edge Cross-Sectional Profiles for Polyp Detection in Colonoscopy , 2014, IEEE Journal of Biomedical and Health Informatics.

[22]  Christopher D. Jensen,et al.  Adenoma detection rate and risk of colorectal cancer and death. , 2014, The New England journal of medicine.

[23]  Kleovoulos Tsourides,et al.  Computer vision and augmented reality in gastrointestinal endoscopy , 2015, Gastroenterology report.

[24]  Kevin A. Henry,et al.  Annual Report to the Nation on the Status of Cancer, 1975-2011, Featuring Incidence of Breast Cancer Subtypes by Race/Ethnicity, Poverty, and State , 2015, Journal of the National Cancer Institute.

[25]  Jung-Hwan Oh,et al.  Polyp-Alert: Near real-time feedback during colonoscopy , 2015, Comput. Methods Programs Biomed..

[26]  Philip Roelandt,et al.  Advanced endoscopic imaging: European Society of Gastrointestinal Endoscopy (ESGE) Technology Review , 2016, Endoscopy.

[27]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[28]  K. Chayama,et al.  Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. , 2016, Gastrointestinal endoscopy.

[29]  G. Fernández-Esparrach,et al.  Exploring the clinical potential of an automatic colonic polyp detection method based on the creation of energy maps , 2016, Endoscopy.

[30]  Nicolas Chapados,et al.  Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model , 2017, Gut.

[31]  Michael Riegler,et al.  Efficient disease detection in gastrointestinal videos – global features versus neural networks , 2017, Multimedia Tools and Applications.

[32]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  G. Tziatzios,et al.  New endoscopes and add-on devices to improve colonoscopy performance , 2017, World journal of gastroenterology.

[34]  S. Kudo,et al.  Computer-aided diagnosis for colonoscopy , 2017, Endoscopy.

[35]  Jonathan H. Chen,et al.  Machine Learning and Prediction in Medicine - Beyond the Peak of Inflated Expectations. , 2017, The New England journal of medicine.

[36]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[37]  Aymeric Histace,et al.  Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results From the MICCAI 2015 Endoscopic Vision Challenge , 2017, IEEE Transactions on Medical Imaging.

[38]  Henry Horng-Shing Lu,et al.  Accurate Classification of Diminutive Colorectal Polyps Using Computer-Aided Analysis. , 2017, Gastroenterology.

[39]  K. Mori,et al.  Real-Time Use of Artificial Intelligence in Identification of Diminutive Polyps During Colonoscopy , 2018, Annals of Internal Medicine.

[40]  Hayato Itoh,et al.  Artificial Intelligence-Assisted Polyp Detection for Colonoscopy: Initial Experience. , 2018, Gastroenterology.

[41]  Arianna Menciassi,et al.  Towards a Computed-Aided Diagnosis System in Colonoscopy: Automatic Polyp Segmentation Using Convolution Neural Networks , 2018, J. Medical Robotics Res..