Automated segmentation of 3D anatomical structures on CT images by using a deep convolutional network based on end-to-end learning approach

We have proposed an end-to-end learning approach that trained a deep convolutional neural network (CNN) for automatic CT image segmentation, which accomplished a voxel-wised multiple classification to directly map each voxel on 3D CT images to an anatomical label automatically. The novelties of our proposed method were (1) transforming the anatomical structures segmentation on 3D CT images into a majority voting of the results of 2D semantic image segmentation on a number of 2D-slices from different image orientations, and (2) using “convolution” and “deconvolution” networks to achieve the conventional “coarse recognition” and “fine extraction” functions which were integrated into a compact all-in-one deep CNN for CT image segmentation. The advantage comparing to previous works was its capability to accomplish real-time image segmentations on 2D slices of arbitrary CT-scan-range (e.g. body, chest, abdomen) and produced correspondingly-sized output. In this paper, we propose an improvement of our proposed approach by adding an organ localization module to limit CT image range for training and testing deep CNNs. A database consisting of 240 3D CT scans and a human annotated ground truth was used for training (228 cases) and testing (the remaining 12 cases). We applied the improved method to segment pancreas and left kidney regions, respectively. The preliminary results showed that the accuracies of the segmentation results were improved significantly (pancreas was 34% and kidney was 8% increased in Jaccard index from our previous results). The effectiveness and usefulness of proposed improvement for CT image segmentations were confirmed.

[1]  Marius George Linguraru,et al.  Abdominal multi-organ segmentation from CT images using conditional shape-location and unsupervised intensity priors , 2015, Medical Image Anal..

[2]  Akinobu Shimizu,et al.  Segmentation of multiple organs in non-contrast 3D abdominal CT images , 2007, International Journal of Computer Assisted Radiology and Surgery.

[3]  Daniel Rueckert,et al.  Automated Abdominal Multi-Organ Segmentation With Subject-Specific Atlas Generation , 2013, IEEE Transactions on Medical Imaging.

[4]  Weimin Huang,et al.  3D-SIFT-Flow for atlas-based CT liver image segmentation. , 2016, Medical physics.

[5]  Hans-Peter Meinzer,et al.  Statistical shape models for 3D medical image segmentation: A review , 2009, Medical Image Anal..

[6]  Song Wang,et al.  Automatic localization of solid organs on 3D CT images by a collaborative majority voting decision based on ensemble learning , 2012, Comput. Medical Imaging Graph..

[7]  Jerry L Prince,et al.  Current methods in medical image segmentation. , 2000, Annual review of biomedical engineering.

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Lubomir M. Hadjiiski,et al.  Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets. , 2016, Medical physics.

[10]  Hiroyuki Yoshida,et al.  Deep transfer learning of virtual endoluminal views for the detection of polyps in CT colonography , 2016, SPIE Medical Imaging.

[11]  Ronald M. Summers,et al.  Deep convolutional networks for pancreas segmentation in CT imaging , 2015, Medical Imaging.

[12]  Xiangrong Zhou,et al.  Automatic anatomy partitioning of the torso region on CT images by using multiple organ localizations with a group-wise calibration technique , 2015, Medical Imaging.

[13]  Giovanni Montana,et al.  Deep neural networks for anatomical brain segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[15]  Nathan Lay,et al.  Rapid Multi-organ Segmentation Using Context Integration and Discriminative Models , 2013, IPMI.

[16]  Jayaram K. Udupa,et al.  Automatic thoracic anatomy segmentation on CT images using hierarchical fuzzy models and registration , 2014, Medical Imaging.

[17]  Xinjian Chen,et al.  Joint segmentation of anatomical and functional images: Applications in quantification of lesions from PET, PET-CT, MRI-PET, and MRI-PET-CT images , 2013, Medical Image Anal..

[18]  Kunio Doi,et al.  Computer-aided diagnosis in medical imaging: Historical review, current status and future potential , 2007, Comput. Medical Imaging Graph..

[19]  Song Wang,et al.  First trial and evaluation of anatomical structure segmentations in 3D CT images based only on deep learning , 2016 .

[20]  Hiroshi Fujita,et al.  Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique. , 2016, Medical physics.

[21]  Bram van Ginneken,et al.  Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box , 2015, Medical Image Anal..

[22]  Song Wang,et al.  Three-Dimensional CT Image Segmentation by Combining 2D Fully Convolutional Network with 3D Majority Voting , 2016, LABELS/DLMIA@MICCAI.