Efficient 3D Fully Convolutional Networks for Pulmonary Lobe Segmentation in CT Images

The human lung is a complex respiratory organ, consisting of five distinct anatomic compartments called lobes. Accurate and automatic segmentation of these pulmonary lobes from computed tomography (CT) images is of clinical importance for lung disease assessment and treatment planning. However, this task is challenging due to ambiguous lobar boundaries, anatomical variations and pathological deformations. In this paper, we propose a high-resolution and efficient 3D fully convolutional network to automatically segment the lobes. We refer to the network as Pulmonary Lobe Segmentation Network (PLS-Net), which is designed to efficiently exploit 3D spatial and contextual information from high-resolution volumetric CT images for effective volume-to-volume learning and inference. The PLS-Net is based on an asymmetric encoder-decoder architecture with three novel components: (i) 3D depthwise separable convolutions to improve the network efficiency by factorising each regular 3D convolution into two simpler operations; (ii) dilated residual dense blocks to efficiently expand the receptive field of the network and aggregate multi-scale contextual information for segmentation; and (iii) input reinforcement at each downsampled resolution to compensate for the loss of spatial information due to convolutional and downsampling operations. We evaluated the proposed PLS-Net on a multi-institutional dataset that consists of 210 CT images acquired from patients with a wide range of lung abnormalities. Experimental results show that our PLS-Net achieves state-of-the-art performance with better computational efficiency. Further experiments confirm the effectiveness of each novel component of the PLS-Net.

[1]  David Gavaghan,et al.  Review of automatic pulmonary lobe segmentation methods from CT , 2015, Comput. Medical Imaging Graph..

[2]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  S. Nemec,et al.  Upper lobe-predominant diseases of the lung. , 2013, AJR. American journal of roentgenology.

[4]  T. Wilkinson,et al.  Present and future utility of computed tomography scanning in the assessment and management of COPD , 2016, European Respiratory Journal.

[5]  Hao Wu,et al.  Mixed Precision Training , 2017, ICLR.

[6]  Adam P. Harrison,et al.  Pathological Pulmonary Lobe Segmentation from CT Images Using Progressive Holistically Nested Neural Networks and Random Walker , 2017, DLMIA/ML-CDS@MICCAI.

[7]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Eric A. Hoffman,et al.  Atlas-driven lung lobe segmentation in volumetric X-ray CT images , 2006, IEEE Transactions on Medical Imaging.

[9]  Gary E. Christensen,et al.  FissureNet: A Deep Learning Approach For Pulmonary Fissure Detection in CT Images , 2019, IEEE Transactions on Medical Imaging.

[10]  Tahreema N. Matin,et al.  Chronic Obstructive Pulmonary Disease: Lobar Analysis with Hyperpolarized 129Xe MR Imaging. , 2017, Radiology.

[11]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[12]  Nima Tajbakhsh,et al.  Automatic Segmentation of Pulmonary Lobes Using a Progressive Dense V-Network , 2018, DLMIA/ML-CDS@MICCAI.

[13]  H. Peitgen,et al.  Informatics in radiology (infoRAD): new tools for computer assistance in thoracic CT. Part 1. Functional analysis of lungs, lung lobes, and bronchopulmonary segments. , 2005, Radiographics : a review publication of the Radiological Society of North America, Inc.

[14]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[15]  Bram van Ginneken,et al.  Automatic Segmentation of the Pulmonary Lobes From Chest CT Scans Based on Fissures, Vessels, and Bronchi , 2013, IEEE Transactions on Medical Imaging.

[16]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[17]  Richard C. Pais,et al.  The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. , 2011, Medical physics.

[18]  Shuicheng Yan,et al.  Dual Path Networks , 2017, NIPS.

[19]  Michael Pienn,et al.  Pulmonary Lobe Segmentation in CT Images using Alpha-Expansion , 2018, VISIGRAPP.

[20]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[21]  David Gavaghan,et al.  Pulmonary lobe segmentation from CT images using fissureness, airways, vessels and multilevel B-splines , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[22]  Heinz Handels,et al.  Pulmonary lobe segmentation with level sets , 2012, Medical Imaging.

[23]  Richard A. Robb,et al.  Processing of CT images for analysis of diffuse lung disease in the lung tissue research consortium , 2008, SPIE Medical Imaging.

[24]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[25]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Joseph M. Reinhardt,et al.  Anatomy-Guided Lung Lobe Segmentation in X-Ray CT Images , 2009, IEEE Transactions on Medical Imaging.

[27]  Kilian Q. Weinberger,et al.  Memory-Efficient Implementation of DenseNets , 2017, ArXiv.

[28]  David Gur,et al.  Pulmonary Lobe Segmentation in CT Examinations Using Implicit Surface Fitting , 2009, IEEE Transactions on Medical Imaging.

[29]  Suicheng Gu,et al.  Robust pulmonary lobe segmentation against incomplete fissures , 2012, Medical Imaging.

[30]  Tahreema N. Matin,et al.  Learning-Based Multi-atlas Segmentation of the Lungs and Lobes in Proton MR Images , 2017, MICCAI.

[31]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[32]  S. Nemec,et al.  Lower lobe-predominant diseases of the lung. , 2013, AJR. American journal of roentgenology.

[33]  Serge J. Belongie,et al.  Residual Networks Behave Like Ensembles of Relatively Shallow Networks , 2016, NIPS.

[34]  Wei Liu,et al.  ParseNet: Looking Wider to See Better , 2015, ArXiv.

[35]  Bram van Ginneken,et al.  Automatic Segmentation of Pulmonary Segments From Volumetric Chest CT Scans , 2009, IEEE Transactions on Medical Imaging.

[36]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[38]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[39]  Edwin K Silverman,et al.  Pulmonary lobe segmentation based on ridge surface sampling and shape model fitting. , 2013, Medical physics.

[40]  Ronald M. Summers,et al.  Progressive and Multi-path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images , 2017, MICCAI.

[41]  Joon Beom Seo,et al.  Fully Automated Lung Lobe Segmentation in Volumetric Chest CT with 3D U-Net: Validation with Intra- and Extra-Datasets , 2019, Journal of Digital Imaging.

[42]  Andreas Fouras,et al.  Quantification of heterogeneity in lung disease with image-based pulmonary function testing , 2016, Scientific Reports.

[43]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[44]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[45]  Adrian Galdran,et al.  End-to-End Supervised Lung Lobe Segmentation , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[46]  David J. Hawkes,et al.  Pulmonary Lobe Segmentation With Probabilistic Segmentation of the Fissures and a Groupwise Fissure Prior , 2017, IEEE Transactions on Medical Imaging.