Two-stream collaborative network for multi-label chest X-ray Image classification with lung segmentation

Abstract Automated chest X-ray (CXR) image analysis is often subject to serious disruption and misguided by its imaging artifacts and noise regions. To minimize such negative effects, many state-of-the-art works have made great efforts in the precise segmentation of lung fields. Based on these works, some image-based features can be extracted directly from lung filed to provide clues for many types of lung diseases such as lung nodule, cardiomegaly, pneumothorax, or emphysema. In this paper, we propose a novel two-stream collaborative network call TSCN for multi-label CXR image classification based on lung segmentation. Specifically, we first train a robust lung segmentor with U-Net and apply it to capture the lung filed from the original CXR image. Then we perform a two-stream feature fusion operation to aggregate the contextual information in both the global image and lung field for complementary feature learning. By taking advantage of the two-stream deep structures and two types of image inputs, a novel self-adaptive weighted fusion scheme is designed to jointly learn these two feature streams in the end-to-end training phase, in order to realize adaptive two-stream feature subsets selection and optimization. Extensive experiments on the ChestX-ray14 dataset demonstrate the effectiveness of the proposed method as compared with the state-of-the-art baselines.

[1]  Sameer Antani,et al.  Medical Imaging : Artificial Intelligence, Image Recognition, and Machine Learning Techniques , 2019 .

[2]  Shuihua Wang,et al.  Cerebral Micro-Bleeding Detection Based on Densely Connected Neural Network , 2019, Front. Neurosci..

[3]  Wei Zeng,et al.  Deep Learning with Lung Segmentation and Bone Shadow Exclusion Techniques for Chest X-Ray Analysis of Lung Cancer , 2017, ArXiv.

[4]  K. Doi,et al.  Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists' detection of pulmonary nodules. , 2000, AJR. American journal of roentgenology.

[5]  Xiaofeng Zhu,et al.  Efficient Utilization of Missing Data in Cost-Sensitive Learning , 2019, IEEE Transactions on Knowledge and Data Engineering.

[6]  M. Kallergi,et al.  Improved method for automatic identification of lung regions on chest radiographs. , 2001 .

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[9]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Dorin Comaniciu,et al.  Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks , 2018, CIARP.

[11]  Lina Yao,et al.  Diagnosis Code Assignment Using Sparsity-Based Disease Correlation Embedding , 2016, IEEE Transactions on Knowledge and Data Engineering.

[12]  Yi Yang,et al.  Semantic Pooling for Complex Event Analysis in Untrimmed Videos , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jie Liu,et al.  Detecting cerebral microbleeds with transfer learning , 2019, Machine Vision and Applications.

[14]  Yi Yang,et al.  Action recognition by exploring data distribution and feature correlation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Stefan Jaeger,et al.  Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. , 2014, Quantitative imaging in medicine and surgery.

[16]  Xavier Robin,et al.  pROC: an open-source package for R and S+ to analyze and compare ROC curves , 2011, BMC Bioinformatics.

[17]  Tao Xu,et al.  An edge-region force guided active shape approach for automatic lung field detection in chest radiographs , 2012, Comput. Medical Imaging Graph..

[18]  Jinxing Li,et al.  DualCheXNet: dual asymmetric feature learning for thoracic disease classification in chest X-rays , 2019, Biomed. Signal Process. Control..

[19]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Yuxing Tang,et al.  Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs , 2018, MLMI@MICCAI.

[21]  Yu-Dong Zhang,et al.  Chinese Sign Language Fingerspelling via Six-Layer Convolutional Neural Network with Leaky Rectified Linear Units for Therapy and Rehabilitation , 2019, J. Medical Imaging Health Informatics.

[22]  Qinghua Zheng,et al.  An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition , 2018, IEEE Transactions on Cybernetics.

[23]  Yi Yang,et al.  Semi-Supervised Multiple Feature Analysis for Action Recognition , 2014, IEEE Transactions on Multimedia.

[24]  Bin Liu,et al.  Unilateral sensorineural hearing loss identification based on double-density dual-tree complex wavelet transform and multinomial logistic regression , 2019, Integr. Comput. Aided Eng..

[25]  David Zhang,et al.  Label Co-Occurrence Learning With Graph Convolutional Networks for Multi-Label Chest X-Ray Image Classification , 2020, IEEE Journal of Biomedical and Health Informatics.

[26]  A. Kalinovsky,et al.  Lung image Ssgmentation using deep learning methods and convolutional neural networks , 2016 .

[27]  Yan Shen,et al.  Dynamic Routing on Deep Neural Network for Thoracic Disease Classification and Sensitive Area Localization , 2018, MLMI@MICCAI.

[28]  Yaping Huang,et al.  Multi-label chest X-ray image classification via category-wise residual attention learning , 2020, Pattern Recognit. Lett..

[29]  David Zhang,et al.  Lesion Location Attention Guided Network for Multi-Label Thoracic Disease Classification in Chest X-Rays , 2019, IEEE Journal of Biomedical and Health Informatics.

[30]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[31]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[32]  Junding Sun,et al.  High Performance Multiple Sclerosis Classification by Data Augmentation and AlexNet Transfer Learning Model , 2019, J. Medical Imaging Health Informatics.

[33]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[34]  Anselmo Cardoso de Paiva,et al.  An automatic method for lung segmentation and reconstruction in chest X-ray using deep neural networks , 2019, Comput. Methods Programs Biomed..

[35]  Adam P. Harrison,et al.  Iterative Attention Mining for Weakly Supervised Thoracic Disease Pattern Localization in Chest X-Rays , 2018, MICCAI.

[36]  Chien-Cheng Lee,et al.  Chest X-Ray Image Segmentation Using Encoder-Decoder Convolutional Network , 2018, 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW).