Classification of glomerular pathological findings using deep learning and nephrologist-AI collective intelligence approach

Background: Automated classification of glomerular pathological findings is potentially beneficial in establishing an efficient and objective diagnosis in renal pathology. While previous studies have verified the artificial intelligence (AI) models for the classification of global sclerosis and glomerular cell proliferation, there are several other glomerular pathological findings required for diagnosis, and the comprehensive models for the classification of these major findings have not yet been reported. Whether the cooperation between these AI models and clinicians improves diagnostic performance also remains unknown. Here, we developed AI models to classify glomerular images for major findings required for pathological diagnosis and investigated whether those models could improve the diagnostic performance of nephrologists. Methods: We used a dataset of 283 kidney biopsy cases comprising 15888 glomerular images that were annotated by a total of 25 nephrologists. AI models to classify seven pathological findings: global sclerosis, segmental sclerosis, endocapillary proliferation, mesangial matrix accumulation, mesangial cell proliferation, crescent, and basement membrane structural changes, were constructed using deep learning by fine-tuning of InceptionV3 convolutional neural network. Subsequently, we compared the agreement to truth labels between majority decision among nephrologists with or without the AI model as a voter. Results: Our model for global sclerosis showed high performance (area under the curve: periodic acid-Schiff, 0.986; periodic acid methenamine silver, 0.983); the models for the other findings also showed performance close to those of nephrologists. By adding the AI model output to majority decision among nephrologists, the sensitivity and specificity were significantly improved in 9 of 14 constructed models compared to those of nephrologists alone. Conclusion: Our study showed a proof-of-concept for the classification of multiple glomerular findings in a comprehensive method of deep learning and suggested its potential effectiveness in improving diagnostic accuracy of clinicians.

[1]  Michael Gadermayr,et al.  Do We Need Large Annotated Training Data for Detection Applications in Biomedical Imaging? A Case Study in Renal Glomeruli Detection , 2016, MLMI@MICCAI.

[2]  Gloria Bueno,et al.  Glomerulus Classification and Detection Based on Convolutional Neural Networks , 2018, J. Imaging.

[3]  F. Cabitza,et al.  Unintended Consequences of Machine Learning in Medicine , 2017, JAMA.

[4]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  S. Joshua Swamidass,et al.  Deep Learning Global Glomerulosclerosis in Transplant Kidney Frozen Sections , 2018, bioRxiv.

[6]  Susan M Sheehan,et al.  Automatic glomerular identification and quantification of histological phenotypes using image analysis and machine learning. , 2018, American journal of physiology. Renal physiology.

[7]  Bonnie E. Shook-Sa,et al.  . CC-BY-NC-ND 4 . 0 International licenseIt is made available under a is the author / funder , who has granted medRxiv a license to display the preprint in perpetuity , 2021 .

[8]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[9]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[10]  Rabi Yacoub,et al.  Computational Segmentation and Classification of Diabetic Glomerulosclerosis. , 2019, Journal of the American Society of Nephrology : JASN.

[11]  Yoshihiro Hirohashi,et al.  Segmental HOG: new descriptor for glomerulus detection in kidney microscopy image , 2015, BMC Bioinformatics.

[12]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[13]  Daniel A Beard,et al.  Region-Based Convolutional Neural Nets for Localization of Glomeruli in Trichrome-Stained Whole Kidney Sections. , 2018, Journal of the American Society of Nephrology : JASN.

[14]  Bart Smeets,et al.  Deep Learning-Based Histopathologic Assessment of Kidney Tissue. , 2019, Journal of the American Society of Nephrology : JASN.

[15]  Kazuhiko Ohe,et al.  Faster R-CNN-Based Glomerular Detection in Multistained Human Whole Slide Images , 2018, J. Imaging.

[16]  Michael Gadermayr,et al.  CNN cascades for segmenting sparse objects in gigapixel whole slide images , 2019, Comput. Medical Imaging Graph..

[17]  Koshiro Fukiyama,et al.  Outcome study of renal biopsy patients in Okinawa, Japan. , 2004, Kidney international.

[18]  John E. Tomaszewski,et al.  Automated renal histopathology: digital extraction and quantification of renal pathology , 2016, SPIE Medical Imaging.

[19]  Rabi Yacoub,et al.  Multi-radial LBP Features as a Tool for Rapid Glomerular Detection and Assessment in Whole Slide Histopathology Images , 2017, Scientific Reports.

[20]  P. Lakhani,et al.  Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks. , 2017, Radiology.

[21]  Priyamvada Singh,et al.  Association of Pathological Fibrosis With Renal Survival Using Deep Neural Networks , 2018, Kidney international reports.

[22]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[23]  Masahiro Ishikawa,et al.  Extraction of glomeruli in whole slide imaging of kidney biopsy specimens , 2017, Medical Imaging.

[24]  Vijaya B. Kolachalama,et al.  Segmentation of Glomeruli Within Trichrome Images Using Deep Learning , 2018, bioRxiv.

[25]  Hailong Zhu,et al.  Predicting protein functions using incomplete hierarchical labels , 2015, BMC Bioinformatics.

[26]  Yan Zhao,et al.  Automatic glomerulus extraction in whole slide images towards computer aided diagnosis , 2016, 2016 IEEE 12th International Conference on e-Science (e-Science).

[27]  Anurag Gupta,et al.  Deep neural network improves fracture detection by clinicians , 2018, Proceedings of the National Academy of Sciences.

[28]  Jinglu Hu,et al.  Glomerulus extraction by using genetic algorithm for edge patching , 2009, 2009 IEEE Congress on Evolutionary Computation.

[29]  Jun Zhang,et al.  Glomerulus Extraction by Optimizing the Fitting Curve , 2008, 2008 International Symposium on Computational Intelligence and Design.

[30]  John E. Tomaszewski,et al.  Automatic computational labeling of glomerular textural boundaries , 2017, Medical Imaging.

[31]  Angelo Duarte,et al.  PathoSpotter-K: A computational tool for the automatic identification of glomerular lesions in histological images of kidneys , 2017, Scientific Reports.

[32]  Charlene Liew.  The future of radiology augmented with Artificial Intelligence: A strategy for success. , 2018, European journal of radiology.

[33]  Jean-Christophe Olivo-Marin,et al.  An approach for detection of glomeruli in multisite digital pathology , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[34]  Rabi Yacoub,et al.  Unsupervised labeling of glomerular boundaries using Gabor filters and statistical testing in renal histology , 2017, Journal of medical imaging.

[35]  Michael Lawrence Barnett,et al.  Comparative Accuracy of Diagnosis by Collective Intelligence of Multiple Physicians vs Individual Physicians , 2019, JAMA network open.

[36]  Ralf H. J. M. Kurvers,et al.  Collective Intelligence Meets Medical Decision-Making: The Collective Outperforms the Best Radiologist , 2015, PloS one.

[37]  Vitoantonio Bevilacqua,et al.  An innovative neural network framework to classify blood vessels and tubules based on Haralick features evaluated in histological images of kidney biopsy , 2017, Neurocomputing.

[38]  Kevin W Eliceiri,et al.  NIH Image to ImageJ: 25 years of image analysis , 2012, Nature Methods.

[39]  Michael Gadermayr,et al.  Segmenting renal whole slide images virtually without training data , 2017, Comput. Biol. Medicine.

[40]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[41]  Germain Forestier,et al.  Detection of glomeruli in renal pathology by mutual comparison of multiple staining modalities , 2017, Proceedings of the 10th International Symposium on Image and Signal Processing and Analysis.

[42]  Namkug Kim,et al.  A Fully Automated System Using A Convolutional Neural Network to Predict Renal Allograft Rejection: Extra-validation with Giga-pixel Immunostained Slides , 2019, Scientific Reports.

[43]  Joachim M. Buhmann,et al.  Computational Pathology: Challenges and Promises for Tissue Analysis , 2015, Comput. Medical Imaging Graph..

[44]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.