Searching for pneumothorax in x-ray images using autoencoded deep features

Fast diagnosis and treatment of pneumothorax, a collapsed or dropped lung, is crucial to avoid fatalities. Pneumothorax is typically detected on a chest X-ray image through visual inspection by experienced radiologists. However, the detection rate is quite low due to the complexity of visual inspection for small lung collapses. Therefore, there is an urgent need for automated detection systems to assist radiologists. Although deep learning classifiers generally deliver high accuracy levels in many applications, they may not be useful in clinical practice due to the lack of high-quality and representative labeled image sets. Alternatively, searching in the archive of past cases to find matching images may serve as a “virtual second opinion” through accessing the metadata of matched evidently diagnosed cases. To use image search as a triaging or diagnosis assistant, we must first tag all chest X-ray images with expressive identifiers, i.e., deep features. Then, given a query chest X-ray image, the majority vote among the top k retrieved images can provide a more explainable output. In this study, we searched in a repository with more than 550,000 chest X-ray images. We developed the Autoencoding Thorax Net (short AutoThorax -Net) for image search in chest radiographs. Experimental results show that image search based on AutoThorax -Net features can achieve high identification performance providing a path towards real-world deployment. We achieved 92% AUC accuracy for a semi-automated search in 194,608 images (pneumothorax and normal) and 82% AUC accuracy for fully automated search in 551,383 images (normal, pneumothorax and many other chest diseases).

[1]  K. Park,et al.  Effective Diagnosis and Treatment through Content-Based Medical Image Retrieval (CBMIR) by Using Artificial Intelligence , 2019, Journal of clinical medicine.

[2]  Yiheng Cai,et al.  Medical image retrieval based on the deep convolution network and hash coding , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[3]  Ashequl Qadir,et al.  Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks , 2018, ArXiv.

[4]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[5]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Laurence Parker,et al.  Utilization Trends in Noncardiac Thoracic Imaging, 2002-2014. , 2017, Journal of the American College of Radiology : JACR.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Jared A. Dunnmon,et al.  Assessment of Convolutional Neural Networks for Automated Classification of Chest Radiographs. , 2019, Radiology.

[9]  Michael Grass,et al.  Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification , 2018, Scientific Reports.

[10]  Yiyu Cai,et al.  Deep Learning for Chest Radiology: A Review , 2019, Current Radiology Reports.

[11]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[12]  Charles E Kahn,et al.  Actionable findings and the role of IT support: report of the ACR Actionable Reporting Work Group. , 2014, Journal of the American College of Radiology : JACR.

[13]  Nathan Cross,et al.  Deep Learning for Pneumothorax Detection and Localization in Chest Radiographs , 2019, ArXiv.

[14]  Yi Yang,et al.  Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification , 2018, ArXiv.

[15]  Antonio Sze-To,et al.  tCheXNet: Detecting Pneumothorax on Chest X-Ray Images Using Deep Transfer Learning , 2019, ICIAR.

[16]  Anastasios Tefas,et al.  Deep convolutional learning for Content Based Image Retrieval , 2018, Neurocomputing.

[17]  Hamid R. Tizhoosh,et al.  Medical Image Classification via SVM Using LBP Features from Saliency-Based Folded Data , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[18]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[19]  Wei Liu,et al.  Towards Large-Scale Histopathological Image Analysis: Hashing-Based Image Retrieval , 2015, IEEE Transactions on Medical Imaging.

[20]  B. Hamm,et al.  Comparing different deep learning architectures for classification of chest radiographs , 2020, Scientific Reports.

[21]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[22]  Qi Tian,et al.  Recent Advance in Content-based Image Retrieval: A Literature Survey , 2017, ArXiv.

[23]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[24]  Stefan Holban,et al.  A Survey of Distance Metrics in Clustering Data Mining Techniques , 2019, Proceedings of the 2019 3rd International Conference on Graphics and Signal Processing.

[25]  A. Ng,et al.  Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists , 2018, PLoS medicine.

[26]  Eoin C Kavanagh,et al.  The Development of Expertise in Radiology: In Chest Radiograph Interpretation, "Expert" Search Pattern May Predate "Expert" Levels of Diagnostic Accuracy for Pneumothorax Identification. , 2016, Radiology.

[27]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[28]  P. Xie,et al.  COVID-CT-Dataset: A CT Scan Dataset about COVID-19 , 2020, ArXiv.

[29]  Ujjwal Maulik,et al.  Deep Learning for Detection and Localization of Thoracic Diseases Using Chest X-Ray Imagery , 2019, ICAISC.

[30]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[31]  Marcus A. Badgeley,et al.  Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study , 2018, PLoS medicine.

[32]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[33]  Dorin Comaniciu,et al.  Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks , 2018, CIARP.

[34]  Tuan D Pham.  A comprehensive study on classification of COVID-19 on computed tomography with pretrained convolutional neural networks , 2020, Scientific reports.

[35]  Roger G. Mark,et al.  MIMIC-CXR: A large publicly available database of labeled chest radiographs , 2019, ArXiv.

[36]  Arambam Neelima,et al.  An overview of approaches for content-based medical image retrieval , 2017, International Journal of Multimedia Information Retrieval.

[37]  Li Yao,et al.  Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions , 2018, ArXiv.

[38]  Muhammad Awais,et al.  Medical image retrieval using deep convolutional neural network , 2017, Neurocomputing.

[39]  Wei-Hung Weng,et al.  Learning Deep Representations of Medical Images using Siamese CNNs with Application to Content-Based Image Retrieval , 2017, ArXiv.

[40]  Ruoyu Li,et al.  Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays , 2018, BCB.

[41]  Yifan Yu,et al.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison , 2019, AAAI.

[42]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[43]  J. Mongan,et al.  Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: A retrospective study , 2018, PLoS medicine.

[44]  Konstantinos Zarogoulidis,et al.  Pneumothorax: from definition to diagnosis and treatment. , 2014, Journal of thoracic disease.

[45]  Li Tong,et al.  Mitigating the Effect of Dataset Bias on Training Deep Models for Chest X-rays , 2019, ArXiv.

[46]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[47]  Daniel Fabbri,et al.  Toward content-based image retrieval with deep convolutional neural networks , 2015, Medical Imaging.

[48]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[49]  Yuan Luo,et al.  Deep Generative Classifiers for Thoracic Disease Diagnosis with Chest X-ray Images , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).