Multi-projection deep learning network for segmentation of 3D medical images

Abstract Segmentation of three-dimensional (3D) medical images using deep learning is a challenging task due to the lack of a 3D medical image dataset and their ground truth, resource memory limitations, and imbalanced dataset problem. In this paper, we propose advanced deep learning network for segmentation of 3D medical images. The proposed Multi-projection Network can preserve resource memory by applying two-dimensional (2D) kernels while still obtaining the 3D information from the image by incorporating slices from different planar projections of the 3D image to achieve good segmentation results. The proposed network uses a weighted cost function to address the imbalanced dataset problem and introduces an adaptive weight that considers the probability of each class in the image. The experimental results showed that the proposed Multi-projection Network can produce the highest sensitivity (true positive rate) compared to other architectures despite the high class imbalance in the dataset and small amount of training data.

[1]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[2]  Dimitris N. Metaxas,et al.  Automated anatomical landmark detection ondistal femur surface using convolutional neural network , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[3]  Lin Yang,et al.  Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation , 2016, NIPS.

[4]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[5]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[7]  Sebastian J. Schlecht,et al.  Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks , 2017, ArXiv.

[8]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[9]  Jae-Sung Bae,et al.  Deep vector-based convolutional neural network approach for automatic recognition of colonies of induced pluripotent stem cells , 2017, PloS one.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Mark D. McDonnell,et al.  Understanding Data Augmentation for Classification: When to Warp? , 2016, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[12]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[13]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[14]  Ronald M. Summers,et al.  Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation , 2015, IEEE Transactions on Medical Imaging.

[15]  Saeed Hassanpour,et al.  Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans , 2018, Comput. Biol. Medicine.

[16]  Tong Liu,et al.  Segmentation of histological images and fibrosis identification with a convolutional neural network , 2018, Comput. Biol. Medicine.

[17]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[18]  Simon J. Doran,et al.  Stacked Autoencoders for Unsupervised Feature Learning and Multiple Organ Detection in a Pilot Study Using 4D Patient Data , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[20]  Jerry L Prince,et al.  Current methods in medical image segmentation. , 2000, Annual review of biomedical engineering.

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Neeraj Sharma,et al.  Automated medical image segmentation techniques , 2010, Journal of medical physics.

[23]  Wesley De Neve,et al.  Dual Rectified Linear Units (DReLUs): A replacement for tanh activation functions in Quasi-Recurrent Neural Networks , 2018, Pattern Recognit. Lett..

[24]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[25]  Alex Lallement,et al.  Survey on deep learning for radiotherapy , 2018, Comput. Biol. Medicine.

[26]  Klaus H. Maier-Hein,et al.  Deep MRI brain extraction: A 3D convolutional neural network for skull stripping , 2016, NeuroImage.

[27]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[28]  Hong Zhao,et al.  A deep convolutional neural network for classification of red blood cells in sickle cell anemia , 2017, PLoS Comput. Biol..

[29]  Chidchanok Lursinsap,et al.  A modified error function for imbalanced dataset classification problem , 2012, 2012 7th International Conference on Computing and Convergence Technology (ICCCT).

[30]  Longbing Cao,et al.  Training deep neural networks on imbalanced data sets , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[31]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[32]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[33]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[34]  Odelin Charron,et al.  Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network , 2018, Comput. Biol. Medicine.

[35]  Guijin Wang,et al.  LRID: A new metric of multi-class imbalance degree based on likelihood-ratio test , 2018, Pattern Recognit. Lett..

[36]  Lisa Tang,et al.  Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation , 2015, MICCAI.

[37]  Z.J. Koles,et al.  Medical Image Segmentation: Methods and Software , 2007, 2007 Joint Meeting of the 6th International Symposium on Noninvasive Functional Source Imaging of the Brain and Heart and the International Conference on Functional Biomedical Imaging.

[38]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[39]  Dorin Comaniciu,et al.  3D Deep Learning for Efficient and Robust Landmark Detection in Volumetric Data , 2015, MICCAI.

[40]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[41]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[42]  En Zhu,et al.  Deep Clustering with Convolutional Autoencoders , 2017, ICONIP.

[43]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.