Fully Automated Multi-Organ Segmentation in Abdominal Magnetic Resonance Imaging with Deep Neural Networks

PURPOSE Segmentation of multiple organs-at-risk (OARs) is essential for MR-only radiation therapy treatment planning and MR-guided adaptive radiotherapy of abdominal cancers. Current practice requires manual delineation that is labor-intensive, time-consuming, and prone to intra- and inter-observer variations. We developed a deep learning (DL) technique for fully automated segmentation of multiple OARs on clinical abdominal MR images with high accuracy, reliability, and efficiency. METHODS We developed Automated deep Learning-based Abdominal Multi-Organ segmentation (ALAMO) technique based on 2D U-net and a densely connected network structure with tailored design in data augmentation and training procedures such as deep connection, auxiliary supervision, and multi-view. The model takes in multi-slice MR images and generates the output of segmentation results. 3.0-Tesla T1 VIBE (Volumetric Interpolated Breath-hold Examination) images of 102 subjects were used in our study and split into 66 for training, 16 for validation, and 20 for testing. Ten OARs were studied, including the liver, spleen, pancreas, left/right kidneys, stomach, duodenum, small intestine, spinal cord, and vertebral bodies. An experienced radiologist manually labeled each OAR, followed by reediting, if necessary, by a senior radiologist, to create the ground-truth. The performance was measured using volume overlapping and surface distance. RESULTS The ALAMO technique generated segmentation labels in good agreement with the manual results. Specifically, among the 10 OARs, 9 achieved high Dice Similarity Coefficients (DSCs) in the range of 0.87-0.96, except for the duodenum with a DSC of 0.80. The inference completed within one minute for a 3D volume of 320x288x180. Overall, the ALAMO model matched the state-of-the-art techniques in performance. CONCLUSION The proposed ALAMO technique allows for fully automated abdominal MR segmentation with high accuracy and practical memory and computation time demands.

[1]  Masahiro Jinzaki,et al.  Semantic Segmentation of Thigh Muscle using 2.5D Deep Learning Network Trained with Limited Datasets , 2019, ArXiv.

[2]  Barbara Villarini,et al.  Advancing Pancreas Segmentation in Multi-protocol MRI Volumes Using Hausdorff-Sine Loss Function , 2019, MLMI@MICCAI.

[3]  Xiaowei Ding,et al.  Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation , 2019, Medical Image Anal..

[4]  Joey Nicholson,et al.  Automated Segmentation of Tissues Using CT and MRI: A Systematic Review. , 2019, Academic radiology.

[5]  Xiaoliang Gong,et al.  Computer-aided pancreas segmentation based on 3D GRE Dixon MRI: a feasibility study , 2019, Acta radiologica open.

[6]  Dorin Comaniciu,et al.  Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Yonggang Lu,et al.  A novel MRI segmentation method using CNN‐based correction network for MRI‐guided adaptive radiotherapy , 2018, Medical physics.

[8]  H. Chandarana,et al.  Emerging role of MRI in radiation therapy , 2018, Journal of magnetic resonance imaging : JMRI.

[9]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[10]  Yan Wang,et al.  Abdominal multi-organ segmentation with organ-attention networks and statistical fusion , 2018, Medical Image Anal..

[11]  Carlo Luschi,et al.  Revisiting Small Batch Training for Deep Neural Networks , 2018, ArXiv.

[12]  Debiao Li,et al.  Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network , 2018, MICCAI.

[13]  Shunxing Bao,et al.  Fully convolutional neural networks improve abdominal organ segmentation , 2018, Medical Imaging.

[14]  Carri K Glide-Hurst,et al.  MRI-only treatment planning: benefits and challenges , 2018, Physics in medicine and biology.

[15]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[16]  Mateusz Buda,et al.  Deep learning in radiology: An overview of the concepts and a survey of the state of the art with focus on MRI , 2018, Journal of magnetic resonance imaging : JMRI.

[17]  Feng Shi,et al.  Brain MRI super resolution using 3D deep densely connected neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[18]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[19]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[20]  Le Lu,et al.  Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function , 2017, ArXiv.

[21]  Daniel L. Rubin,et al.  Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions , 2017, Journal of Digital Imaging.

[22]  E S Paulson,et al.  Recommendations for MRI-based contouring of gross tumor volume and organs at risk for radiation therapy of pancreatic cancer. , 2017, Practical radiation oncology.

[23]  G. Litjens,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[24]  Nico Karssemeijer,et al.  Using deep learning to segment breast and fibroglandular tissue in MRI volumes , 2017, Medical physics.

[25]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Le Lu,et al.  Pancreas Segmentation in MRI Using Graph-Based Decision Fusion on Convolutional Neural Networks , 2016, MICCAI.

[27]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[29]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[30]  Andy Davis,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[31]  Hamid Jafarkhani,et al.  A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI , 2015, Medical Image Anal..

[32]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[35]  Mert R. Sabuncu,et al.  Multi-atlas segmentation of biomedical images: A survey , 2014, Medical Image Anal..

[36]  X Allen Li,et al.  Variability of target and normal structure delineation using multimodality imaging for radiation therapy of pancreatic cancer. , 2014, International journal of radiation oncology, biology, physics.

[37]  C. Jilg,et al.  3 Tesla multiparametric MRI for GTV-definition of Dominant Intraprostatic Lesions in patients with Prostate Cancer – an interobserver variability study , 2013, Radiation oncology.

[38]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[39]  S. Dević.  MRI simulation for radiotherapy treatment planning. , 2012, Medical physics.

[40]  Pierre-Marc Jodoin,et al.  3D segmentation of abdominal aorta from CT-scan and MR images , 2012, Comput. Medical Imaging Graph..

[41]  Timo Kohlberger,et al.  Automatic Multi-organ Segmentation Using Learning-Based Segmentation and Level Set Optimization , 2011, MICCAI.

[42]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[43]  N. Rofsky,et al.  Abdominal MR imaging with a volumetric interpolated breath-hold examination. , 1999, Radiology.

[44]  Dong Joong Kang,et al.  A fast and stable snake algorithm for medical images , 1999, Pattern Recognition Letters.

[45]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Daniel Rueckert,et al.  Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II , 2017, Lecture Notes in Computer Science.

[47]  C J Moore,et al.  Automated delineation of radiotherapy volumes: are we going in the right direction? , 2013, The British journal of radiology.

[48]  T. Sørensen,et al.  A method of establishing group of equal amplitude in plant sociobiology based on similarity of species content and its application to analyses of the vegetation on Danish commons , 1948 .

[49]  P. Jaccard,et al.  Etude comparative de la distribution florale dans une portion des Alpes et des Jura , 1901 .