Bone segmentation on whole-body CT using convolutional neural network with novel data augmentation techniques

BACKGROUND The purpose of this study was to develop and evaluate an algorithm for bone segmentation on whole-body CT using a convolutional neural network (CNN). METHODS Bone segmentation was performed using a network based on U-Net architecture. To evaluate its performance and robustness, we prepared three different datasets: (1) an in-house dataset comprising 16,218 slices of CT images from 32 scans in 16 patients; (2) a secondary dataset comprising 12,529 slices of CT images from 20 scans in 20 patients, which were collected from The Cancer Imaging Archive; and (3) a publicly available labelled dataset comprising 270 slices of CT images from 27 scans in 20 patients. To improve the network's performance and robustness, we evaluated the efficacy of three types of data augmentation technique: conventional method, mixup, and random image cropping and patching (RICAP). RESULTS The network trained on the in-house dataset achieved a mean Dice coefficient of 0.983 ± 0.005 on cross validation with the in-house dataset, and 0.943 ± 0.007 with the secondary dataset. The network trained on the public dataset achieved a mean Dice coefficient of 0.947 ± 0.013 on 10 randomly generated 15-3-9 splits of the public dataset. These results outperform those reported previously. Regarding augmentation technique, the conventional method, RICAP, and a combination of these were effective. CONCLUSIONS The CNN-based model achieved accurate bone segmentation on whole-body CT, with generalizability to various scan conditions. Data augmentation techniques enabled construction of an accurate and robust model even with a small dataset.

[1]  T N Hangartner,et al.  Thresholding technique for accurate analysis of density and geometry in QCT, pQCT and microCT images. , 2007, Journal of musculoskeletal & neuronal interactions.

[2]  Steven K Boyd,et al.  Automatic segmentation of cortical and trabecular compartments based on a dual threshold technique for in vivo micro-CT bone analysis. , 2007, Bone.

[3]  Yadong Wang,et al.  Low‐rank and sparse decomposition based shape model and probabilistic atlas for automatic pathological organ segmentation , 2017, Medical Image Anal..

[4]  Nadia Magnenat-Thalmann,et al.  Robust statistical shape models for MRI bone segmentation in presence of small field of view , 2011, Medical Image Anal..

[5]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[6]  M. Jorge Cardoso,et al.  Improving Data Augmentation for Medical Image Segmentation , 2018 .

[7]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[8]  Beat Schmutz,et al.  Effects of CT image segmentation methods on the accuracy of long bone 3D reconstructions. , 2011, Medical engineering & physics.

[9]  Youbao Tang,et al.  CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation , 2018, MICCAI.

[10]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[11]  Zhenwei Zhang,et al.  Radiological images and machine learning: trends, perspectives, and prospects , 2019, Comput. Biol. Medicine.

[12]  Klaus H. Maier-Hein,et al.  Automatic bone segmentation in whole-body CT images , 2018, International Journal of Computer Assisted Radiology and Surgery.

[13]  Sim Heng Ong,et al.  Fast segmentation of bone in CT images using 3D adaptive thresholding , 2010, Comput. Biol. Medicine.

[14]  Gábor Székely,et al.  Fully automatic and fast segmentation of the femur bone from 3D-CT images with no shape prior , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[15]  Wouter M. Kouw,et al.  CT image segmentation of bone for medical additive manufacturing using a convolutional neural network , 2018, Comput. Biol. Medicine.

[16]  J. Torner,et al.  Automated cortical bone segmentation for multirow-detector CT imaging with validation and application to human studies. , 2015, Medical physics.

[17]  Woei-Chyn Chu,et al.  Performance measure characterization for evaluating neuroimage segmentation algorithms , 2009, NeuroImage.

[18]  Takashi Matsubara,et al.  RICAP: Random Image Cropping and Patching Data Augmentation for Deep CNNs , 2018, ACML.

[19]  Ronald M. Summers,et al.  2D View Aggregation for Lymph Node Detection Using a Shallow Hierarchy of Linear Classifiers , 2014, MICCAI.

[20]  Yadong Wang,et al.  A hierarchical local region-based sparse shape composition for liver segmentation in CT scans , 2016, Pattern Recognit..

[21]  M Fiebich,et al.  Automatic bone segmentation technique for CT angiographic studies. , 1999, Journal of computer assisted tomography.

[22]  Hans-Peter Meinzer,et al.  Statistical shape models for 3D medical image segmentation: A review , 2009, Medical Image Anal..

[23]  Xiangjian He,et al.  Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges , 2019, Journal of Digital Imaging.

[24]  Lars Edenbrandt,et al.  Deep learning for segmentation of 49 selected bones in CT scans: First step in automated PET/CT-based 3D quantification of skeletal metastases. , 2019, European journal of radiology.

[25]  Christian S. Perone,et al.  Promises and limitations of deep learning for medical image segmentation , 2019, Journal of Medical Artificial Intelligence.

[26]  José Antonio Pérez-Carrasco,et al.  Joint segmentation of bones and muscles using an intensity and histogram-based energy minimization approach , 2018, Comput. Methods Programs Biomed..

[27]  Ronald M. Summers,et al.  A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations , 2014, MICCAI.

[28]  Jan Wolff,et al.  CT image segmentation methods for bone used in medical additive manufacturing. , 2018, Medical engineering & physics.

[29]  Ronald M. Summers,et al.  Accurate 3D bone segmentation in challenging CT images: Bottom-up parsing and contextualized optimization , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[30]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[31]  Jerry L Prince,et al.  Current methods in medical image segmentation. , 2000, Annual review of biomedical engineering.

[32]  Yadong Wang,et al.  Shape–intensity prior level set combining probabilistic atlas and probability map constrains for automatic liver segmentation from abdominal CT images , 2016, International Journal of Computer Assisted Radiology and Surgery.

[33]  Hans-Christian Hege,et al.  Automatic Segmentation of the Pelvic Bones from CT Data Based on a Statistical Shape Model , 2008, VCBM.

[34]  Punam K. Saha,et al.  Robust segmentation of trabecular bone for in vivo CT imaging using anisotropic diffusion and multi-scale morphological reconstruction , 2017, Medical Imaging.