GAN-based disentanglement learning for chest X-ray rib suppression

Clinical evidence has shown that rib-suppressed chest X-rays (CXRs) can improve the reliability of pulmonary disease diagnosis. However, previous approaches on generating rib-suppressed CXR face challenges in preserving details and eliminating rib residues. We hereby propose a GAN-based disentanglement learning framework called Rib Suppression GAN, or RSGAN, to perform rib suppression by utilizing the anatomical knowledge embedded in unpaired computed tomography (CT) images. In this approach, we employ a residual map to characterize the intensity difference between CXR and the corresponding rib-suppressed result. To predict the residual map in CXR domain, we disentangle the image into structure- and contrast-specific features and transfer the rib structural priors from digitally reconstructed radiographs (DRRs) computed by CT. Furthermore, we employ additional adaptive loss to suppress rib residue and preserve more details. We conduct extensive experiments based on 1673 CT volumes, and four benchmarking CXR datasets, totaling over 120K images, to demonstrate that (i) our proposed RSGAN achieves superior image quality compared to the state-of-the-art rib suppression methods; (ii) combining CXR with our rib-suppressed result leads to better performance in lung disease classification and tuberculosis area detection.

[1]  Nassir Navab,et al.  X-Ray In-Depth Decomposition: Revealing the Latent Structures , 2016, MICCAI.

[2]  Bram van Ginneken,et al.  Clavicle segmentation in chest radiographs , 2012, Medical Image Anal..

[3]  M A Laskey,et al.  Dual-energy X-ray absorptiometry and body composition. , 1996, Nutrition.

[4]  Jaakko Lehtinen,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[6]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[7]  Kenji Suzuki,et al.  Separation of Bones From Chest Radiographs by Means of Anatomically Specific Multiple Massive-Training ANNs Combined With Total Variation Minimization Smoothing , 2014, IEEE Transactions on Medical Imaging.

[8]  Lin Yang,et al.  Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[10]  Qianjin Feng,et al.  Cascade of multi-scale convolutional neural networks for bone suppression of chest radiographs in gradient domain , 2017, Medical Image Anal..

[11]  Bo Zhou,et al.  Generation of Virtual Dual Energy Images from Standard Single-Shot Radiographs using Multi-scale and Conditional Adversarial Network , 2018, ACCV.

[12]  Kunio Doi,et al.  Image-processing technique for suppressing ribs in chest radiographs by means of massive training artificial neural network (MTANN) , 2006, IEEE Transactions on Medical Imaging.

[13]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[15]  Sotirios A. Tsaftaris,et al.  Disentangled representation learning in cardiac image analysis , 2019, Medical Image Anal..

[16]  Jens von Berg,et al.  A novel bone suppression method that improves lung nodule detection , 2015, International Journal of Computer Assisted Radiology and Surgery.

[17]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Fan Zhang,et al.  Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation , 2019, MICCAI.

[19]  Jiannan Wang,et al.  Encoding CT Anatomy Knowledge for Unpaired Chest X-ray Image Decomposition , 2019, MICCAI.

[20]  Peter Maday,et al.  Elimination of clavicle shadows to help automatic lung nodule detection on chest radiographs , 2009 .

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Nassir Navab,et al.  Enabling machine learning in X-ray-based procedures via realistic simulation of image formation , 2019, International Journal of Computer Assisted Radiology and Surgery.

[23]  Hayit Greenspan,et al.  Lung Structures Enhancement in Chest Radiographs via CT Based FCNN Training , 2018, RAMBO+BIA+TIA@MICCAI.

[24]  Qianjin Feng,et al.  Bone Suppression of Chest Radiographs With Cascaded Convolutional Networks in Wavelet Domain , 2019, IEEE Access.

[25]  Jeroen van der Laak,et al.  Residual cyclegan for robust domain transformation of histopathological tissue slides , 2021, Medical Image Anal..

[26]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[27]  Chao Lu,et al.  A learning based deformable template matching method for automatic rib centerline extraction and labeling in CT images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Yun Liu,et al.  Rethinking Computer-Aided Tuberculosis Diagnosis , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Hayit Greenspan,et al.  Improving CNN Training using Disentanglement for Liver Lesion Classification in CT , 2018, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[30]  A. Karargyris,et al.  Automatic screening for tuberculosis in chest radiographs: a survey. , 2013, Quantitative imaging in medicine and surgery.

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Hayit Greenspan,et al.  An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection , 2018, IEEE Journal of Biomedical and Health Informatics.

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Yue Zhang,et al.  Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-ray Image Segmentation , 2018, MICCAI.

[35]  D. Brenner,et al.  Computed tomography--an increasing source of radiation exposure. , 2007, The New England journal of medicine.

[36]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[37]  Brett W Carter,et al.  Missed Lung Cancer. , 2018, Radiologic clinics of North America.

[38]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[39]  Jingjing Lu,et al.  High-Resolution Chest X-Ray Bone Suppression Using Unpaired CT Structural Priors , 2020, IEEE Transactions on Medical Imaging.

[40]  Bram van Ginneken,et al.  Improved texture analysis for automatic detection of tuberculosis (TB) on chest radiographs with bone suppression images , 2013, Medical Imaging.

[41]  Bram van Ginneken,et al.  Suppression of Translucent Elongated Structures: Applications in Chest Radiography , 2013, IEEE Transactions on Medical Imaging.

[42]  Maneesh Kumar Singh,et al.  DRIT++: Diverse Image-to-Image Translation via Disentangled Representations , 2019, International Journal of Computer Vision.

[43]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.

[44]  Stefan Jaeger,et al.  Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. , 2014, Quantitative imaging in medicine and surgery.