Estimation of the Volume of the Left Ventricle From MRI Images Using Deep Neural Networks

Segmenting human left ventricle (LV) in magnetic resonance imaging images and calculating its volume are important for diagnosing cardiac diseases. The latter task became the topic of the Second Annual Data Science Bowl organized by Kaggle. The dataset consisted of a large number of cases with only systole and diastole volume labels. We designed a system based on neural networks to solve this problem. It began with a detector to detect the regions of interest (ROI) containing LV chambers. Then a deep neural network named hypercolumns fully convolutional network was used to segment LV in ROI. The 2-D segmentation results were integrated across different images to estimate the volume. With ground-truth volume labels, this model was trained end-to-end. To improve the result, an additional dataset with only segmentation labels was used. The model was trained alternately on these two tasks. We also proposed a variance estimation method for the final prediction. Our algorithm ranked the fourth on the test set in this competition.

[1]  Daniel Rueckert,et al.  Deep Learning for Cardiac Image Segmentation: A Review , 2020, Frontiers in Cardiovascular Medicine.

[2]  Gustavo Carneiro,et al.  Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance , 2017, Medical Image Anal..

[3]  S. Osher,et al.  Algorithms Based on Hamilton-Jacobi Formulations , 1988 .

[4]  Joseph F. Murray,et al.  Supervised Learning of Image Restoration with Convolutional Networks , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Terry M. Peters,et al.  Left ventricle segmentation in MRI via convex relaxed distribution matching , 2013, Medical Image Anal..

[6]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[8]  Paul F. Whelan,et al.  Segmentation of the Left Ventricle of the Heart in 3-D+t MRI Data Using an Optimized Nonrigid Temporal Model , 2008, IEEE Transactions on Medical Imaging.

[9]  Xiangyang Xu,et al.  Automatic left ventricle segmentation in cardiac MRI using topological stable-state thresholding and region restricted dynamic programming. , 2012, Academic radiology.

[10]  Jens von Berg,et al.  Automated Segmentation of the Left Ventricle in Cardiac MRI , 2003, MICCAI.

[11]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Xiaochun Cao,et al.  Inference With Collaborative Model for Interactive Tumor Segmentation in Medical Image Sequences , 2016, IEEE Transactions on Cybernetics.

[16]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  Milan Sonka,et al.  4-D Cardiac MR Image Analysis: Left and Right Ventricular Morphology and Function , 2010, IEEE Transactions on Medical Imaging.

[18]  Huaifei Hu,et al.  Hybrid segmentation of left ventricle in cardiac MRI using Gaussian-mixture model and region restricted dynamic programming. , 2013, Magnetic resonance imaging.

[19]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Yann LeCun,et al.  Toward automatic phenotyping of developing embryos from videos , 2005, IEEE Transactions on Image Processing.

[23]  Shengcai Liao,et al.  Learning Multi-scale Block Local Binary Patterns for Face Recognition , 2007, ICB.

[24]  Daniel Rueckert,et al.  Atlas-Based Segmentation and Tracking of 3D Cardiac MR Images Using Non-rigid Registration , 2002, MICCAI.

[25]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[26]  Nassir Navab,et al.  Segmentation by retrieval with guided random walks: Application to left ventricle segmentation in MRI , 2013, Medical Image Anal..

[27]  Hamid Jafarkhani,et al.  A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI , 2015, Medical Image Anal..

[28]  H. Sebastian Seung,et al.  Recursive Training of 2D-3D Convolutional Networks for Neuronal Boundary Prediction , 2015, NIPS.

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[31]  Ling Shao,et al.  Spatio-Temporal Laplacian Pyramid Coding for Action Recognition , 2014, IEEE Transactions on Cybernetics.

[32]  Ben M. Herbst,et al.  Left ventricular segmentation from MRI datasets with edge modelling conditional random fields , 2013, BMC Medical Imaging.

[33]  Gustavo Carneiro,et al.  Left ventricle segmentation from cardiac MRI combining level set methods with deep belief networks , 2013, 2013 IEEE International Conference on Image Processing.

[34]  Phi Vu Tran,et al.  A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI , 2016, ArXiv.

[35]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.