Robust Classification from Noisy Labels: Integrating Additional Knowledge for Chest Radiography Abnormality Assessment

Chest radiography is the most common radiographic examination performed in daily clinical practice for the detection of various heart and lung abnormalities. The large amount of data to be read and reported, with more than 100 studies per day for a single radiologist, poses a challenge in consistently maintaining high interpretation accuracy. The introduction of large-scale public datasets has led to a series of novel systems for automated abnormality classification. However, the labels of these datasets were obtained using natural language processed medical reports, yielding a large degree of label noise that can impact the performance. In this study, we propose novel training strategies that handle label noise from such suboptimal data. Prior label probabilities were measured on a subset of training data re-read by 4 board-certified radiologists and were used during training to increase the robustness of the training model to the label noise. Furthermore, we exploit the high comorbidity of abnormalities observed in chest radiography and incorporate this information to further reduce the impact of label noise. Additionally, anatomical knowledge is incorporated by training the system to predict lung and heart segmentation, as well as spatial knowledge labels. To deal with multiple datasets and images derived from various scanners that apply different post-processing techniques, we introduce a novel image normalization strategy. Experiments were performed on an extensive collection of 297,541 chest radiographs from 86,876 patients, leading to a state-of-the-art performance level for 17 abnormalities from 2 datasets. With an average AUC score of 0.880 across all abnormalities, our proposed training strategies can be used to significantly improve performance scores.

[1]  Ashequl Qadir,et al.  Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks , 2018, ArXiv.

[2]  Yanning Zhang,et al.  Triple attention learning for classification of 14 thoracic diseases using chest radiography , 2020, Medical Image Anal..

[3]  J. Leipsic,et al.  The relationship between lung inflammation and cardiovascular disease. , 2012, American journal of respiratory and critical care medicine.

[4]  Wei Wei,et al.  Thoracic Disease Identification and Localization with Limited Supervision , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Khalid Ashraf,et al.  Abnormality Detection and Localization in Chest X-Rays using Deep Convolutional Neural Networks , 2017, ArXiv.

[6]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[7]  K. Hussein A STUDY ON NLP APPLICATIONS AND AMBIGUITY PROBLEMS SHAIDAH JUSOH , 2018 .

[8]  Wei Li,et al.  Pulmonary Nodule Classification with Deep Convolutional Neural Networks on Computed Tomography Images , 2016, Comput. Math. Methods Medicine.

[9]  Andrzej Rusiecki,et al.  Trimmed Robust Loss Function for Training Deep Neural Networks with Label Noise , 2019, ICAISC.

[10]  Dorin Comaniciu,et al.  Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Sabine Dippel,et al.  Multiscale contrast enhancement for radiographies: Laplacian pyramid versus fast wavelet transform , 2002, IEEE Transactions on Medical Imaging.

[12]  Dorin Comaniciu,et al.  Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks , 2018, CIARP.

[13]  A. Amyar,et al.  Multi-task Deep Learning Based CT Imaging Analysis For COVID-19: Classification and Segmentation , 2020, medRxiv.

[14]  Yihong Gong,et al.  Multi-labelled classification using maximum entropy method , 2005, SIGIR '05.

[15]  Michael A. Bruno,et al.  Understanding and Confronting Our Mistakes: The Epidemiology of Error in Radiology and Strategies for Error Reduction. , 2015, Radiographics : a review publication of the Radiological Society of North America, Inc.

[16]  Yaping Huang,et al.  Multi-label chest X-ray image classification via category-wise residual attention learning , 2020, Pattern Recognit. Lett..

[17]  Dmitry P. Vetrov,et al.  Variational Dropout Sparsifies Deep Neural Networks , 2017, ICML.

[18]  Dorin Comaniciu,et al.  Automated detection and quantification of COVID-19 airspace disease on chest radiographs: A novel approach achieving radiologist-level performance using a CNN trained on digital reconstructed radiographs (DRRs) from CT-based ground-truth , 2020, ArXiv.

[19]  Dorin Comaniciu,et al.  Quantifying and Leveraging Classification Uncertainty for Chest Radiograph Assessment , 2019, MICCAI.

[20]  J. Gohagan,et al.  The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial of the National Cancer Institute: history, organization, and status. , 2000, Controlled clinical trials.

[21]  Wei Wang,et al.  Comorbidity and its impact on 1590 patients with COVID-19 in China: a nationwide analysis , 2020, European Respiratory Journal.

[22]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[23]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[24]  Nir Shavit,et al.  Deep Learning is Robust to Massive Label Noise , 2017, ArXiv.

[25]  G. Corrado,et al.  End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography , 2019, Nature Medicine.

[26]  Simon K. Warfield,et al.  Deep learning with noisy labels: exploring techniques and remedies in medical image analysis , 2020, Medical Image Anal..

[27]  Andrew McCallum,et al.  Collective multi-label classification , 2005, CIKM '05.

[28]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[29]  Audun Jøsang,et al.  Subjective Logic: A Formalism for Reasoning Under Uncertainty , 2016 .

[30]  Li Yao,et al.  Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions , 2018, ArXiv.

[31]  Pablo Mesejo,et al.  Deep architectures for high-resolution multi-organ chest X-ray image segmentation , 2019, Neural Computing and Applications.

[32]  M. Kholiavchenko,et al.  Contour-aware multi-label chest X-ray organ segmentation , 2020, International Journal of Computer Assisted Radiology and Surgery.

[33]  Bal'azs Maga,et al.  Attention U-Net Based Adversarial Architectures for Chest X-ray Lung Segmentation information , 2020, ADGN@ECAI.

[34]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[35]  Konstantin Simonov,et al.  Lung boundary detection for chest X-ray images classification based on GLCM and probabilistic neural networks , 2019, KES.

[36]  R. Atun,et al.  Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study , 2005, BMJ : British Medical Journal.

[37]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Vineeth N. Balasubramanian,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[39]  Jeffrey Dean,et al.  Scalable and accurate deep learning with electronic health records , 2018, npj Digital Medicine.

[40]  Li Yao,et al.  Learning to diagnose from scratch by exploiting dependencies among labels , 2017, ArXiv.

[41]  A. Ng,et al.  Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists , 2018, PLoS medicine.

[42]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Enzo Ferrante,et al.  Post-DAE: Anatomically Plausible Segmentation via Post-Processing With Denoising Autoencoders , 2020, IEEE Transactions on Medical Imaging.

[44]  Yi Yang,et al.  Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification , 2018, ArXiv.

[45]  Youngjin Yoo,et al.  Quantifying and Leveraging Predictive Uncertainty for Medical Image Assessment , 2020, Medical Image Anal..

[46]  Akshay Pai,et al.  Lung Segmentation from Chest X-rays using Variational Data Imputation , 2020, ArXiv.

[47]  Lei Wang,et al.  SDFN: Segmentation-based Deep Fusion Network for Thoracic Disease Classification in Chest X-ray Images , 2018, Comput. Medical Imaging Graph..

[48]  Yan Shen,et al.  Dynamic Routing on Deep Neural Network for Thoracic Disease Classification and Sensitive Area Localization , 2018, MLMI@MICCAI.

[49]  Ruoyu Li,et al.  Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays , 2018, BCB.

[50]  Yifan Yu,et al.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison , 2019, AAAI.

[51]  Le Lu,et al.  Thorax-Net: An Attention Regularized Deep Neural Network for Classification of Thoracic Diseases on Chest Radiography , 2020, IEEE Journal of Biomedical and Health Informatics.

[52]  A. Brady Error and discrepancy in radiology: inevitable or avoidable? , 2016, Insights into Imaging.

[53]  Zhiwei Huang,et al.  Fusion High-Resolution Network for Diagnosing ChestX-ray Images , 2020, Electronics.

[54]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[55]  Noel E. O'Connor,et al.  Unsupervised label noise modeling and loss correction , 2019, ICML.

[56]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[57]  Bram van Ginneken,et al.  Localized Energy-Based Normalization of Medical Images: Application to Chest Radiography , 2015, IEEE Transactions on Medical Imaging.

[58]  Xiaohui Xie,et al.  DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[59]  Koray Kavukcuoglu,et al.  Visual Attention , 2020, Computational Models for Cognitive Vision.

[60]  Adam P. Harrison,et al.  Iterative Attention Mining for Weakly Supervised Thoracic Disease Pattern Localization in Chest X-Rays , 2018, MICCAI.

[61]  J. Steiner,et al.  Health and Quality of Life Outcomes , 2003 .

[62]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[63]  Yuxing Tang,et al.  Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs , 2018, MLMI@MICCAI.

[64]  Elyor Kodirov,et al.  IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters , 2019 .

[65]  Mythreyi Bhargavan,et al.  Radiologists' reading times using PACS and using films: one practice's experience. , 2006, Academic radiology.