Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks

Automatic segmentation of the liver and hepatic lesions is an important step towards deriving quantitative biomarkers for accurate clinical diagnosis and computer-aided decision support systems. This paper presents a method to automatically segment liver and lesions in CT and MRI abdomen images using cascaded fully convolutional neural networks (CFCNs) enabling the segmentation of a large-scale medical trial or quantitative image analysis. We train and cascade two FCNs for a combined segmentation of the liver and its lesions. In the first step, we train a FCN to segment the liver as ROI input for a second FCN. The second FCN solely segments lesions within the predicted liver ROIs of step 1. CFCN models were trained on an abdominal CT dataset comprising 100 hepatic tumor volumes. Validations on further datasets show that CFCN-based semantic liver and lesion segmentation achieves Dice scores over 94% for liver with computation times below 100s per volume. We further experimentally demonstrate the robustness of the proposed method on an 38 MRI liver tumor volumes and the public 3DIRCAD dataset.

[1]  Yen-Wei Chen,et al.  Improved segmentation of low-contrast lesions using sigmoid edge model , 2016, International Journal of Computer Assisted Radiology and Surgery.

[2]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[3]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Yong Yin,et al.  A Likelihood and Local Constraint Level Set Model for Liver Tumor Segmentation from CT Volumes , 2013, IEEE Transactions on Biomedical Engineering.

[5]  Hayit Greenspan,et al.  Fully Convolutional Network for Liver Segmentation and Lesions Detection , 2016, LABELS/DLMIA@MICCAI.

[6]  Ghassan Hamarneh,et al.  Spotlight: Automated Confidence-Based User Guidance for Increasing Efficiency in Interactive 3D Image Segmentation , 2010, MCV.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Armando Barreto,et al.  A 3-D Liver Segmentation Method with Parallel Computing for Selective Internal Radiation Therapy , 2012, IEEE Transactions on Information Technology in Biomedicine.

[9]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[10]  Seyed-Ahmad Ahmadi,et al.  Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields , 2016, MICCAI.

[11]  Dani Lischinski,et al.  Liver tumors segmentation from CTA images using voxels classification and affinity constraint propagation , 2011, International Journal of Computer Assisted Radiology and Surgery.

[12]  Ronald M. Summers,et al.  Tumor Burden Analysis on Computed Tomography by Automated Liver and Tumor Segmentation , 2012, IEEE Transactions on Medical Imaging.

[13]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[14]  Riccardo Lencioni,et al.  EASL-EORTC clinical practice guidelines: management of hepatocellular carcinoma. , 2012, Journal of hepatology.

[15]  Hayit Greenspan,et al.  Automated method for detection and segmentation of liver metastatic lesions in follow-up CT examinations , 2015, Journal of medical imaging.

[16]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[17]  Ye Wang,et al.  Liver segmentation with constrained convex variational model , 2014, Pattern Recognit. Lett..

[18]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[19]  Thierry Cresson,et al.  SEMI-automated liver CT segmentation using Laplacian meshes , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[20]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[21]  Ghassan Hamarneh,et al.  Active Learning for Interactive 3D Image Segmentation , 2011, MICCAI.

[22]  Samuel Kadoury,et al.  Metastatic liver tumour segmentation from discriminant Grassmannian manifolds , 2015, Physics in medicine and biology.

[23]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[24]  W. Alexander,et al.  European Association for the Study of the Liver , 1968 .

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[27]  Xinjian Chen,et al.  Automatic Liver Segmentation Based on Shape Constraints and Deformable Graph Cut in CT Images , 2015, IEEE Transactions on Image Processing.

[28]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[29]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[30]  Mika Pollari,et al.  Semi-automatic liver tumor segmentation with hidden Markov measure field model and non-parametric distribution estimation , 2012, Medical Image Anal..

[31]  Hao Chen,et al.  VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation , 2016, ArXiv.

[32]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[33]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[34]  Sebastian J. Schlecht,et al.  SurvivalNet: Predicting patient survival from diffusion weighted magnetic resonance images using cascaded fully convolutional and 3D Convolutional Neural Networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[35]  Martin Styner,et al.  Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets , 2009, IEEE Transactions on Medical Imaging.

[36]  Danny Ziyi Chen,et al.  Detection of Glands and Villi by Collaboration of Domain Knowledge and Deep Learning , 2015, MICCAI.

[37]  Leo Joskowicz,et al.  Automatic Liver Tumor Segmentation in Follow-Up CT Scans: Preliminary Method and Results , 2015, Patch-MI@MICCAI.

[38]  Sandra Morales,et al.  A Method for Liver Segmentation on Computed Tomography Images in Venous Phase Suitable for Real Environments , 2015 .

[39]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  C. Mathers,et al.  Estimates of worldwide burden of cancer in 2008: GLOBOCAN 2008 , 2010, International journal of cancer.

[41]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.