Human-level CMR image analysis with deep fully convolutional networks

Cardiovascular magnetic resonance (CMR) imaging is a standard imaging modality for assessing cardiovascular diseases (CVDs), the leading cause of death globally. CMR enables accurate quantification of the cardiac chamber volume, ejection fraction and myocardial mass, providing a wealth of information for sensitive and specific diagnosis and monitoring of CVDs. However, for years, clinicians have been relying on manual approaches for CMR image analysis, which is time consuming and prone to subjective errors. It is a major clinical challenge to automatically derive quantitative and clinically relevant information from CMR images. Deep neural networks have shown a great potential in image pattern recognition and segmentation for a variety of tasks. Here we demonstrate an automated analysis method for CMR images, which is based on a fully convolutional network (FCN). The network is trained and evaluated on a dataset of unprecedented size, consisting of 4,875 subjects with 93,500 pixelwise annotated images, which is by far the largest annotated CMR dataset. By combining FCN with a large-scale annotated dataset, we show for the first time that an automated method achieves a performance on par with human experts in analysing CMR images and deriving clinical measures. We anticipate this to be a starting point for automated and comprehensive CMR analysis with human-level performance, facilitated by machine learning. It is an important advance on the pathway towards computer-assisted CVD assessment.

[1]  Dudley J Pennell,et al.  Comparison of interstudy reproducibility of cardiovascular magnetic resonance with two-dimensional echocardiography in normal subjects and in patients with heart failure or left ventricular hypertrophy. , 2002, The American journal of cardiology.

[2]  Kieran Clarke,et al.  Journal of Cardiovascular Magnetic Resonance Determinants of Left Ventricular Mass in Obesity; a Cardiovascular Magnetic Resonance Study , 2022 .

[3]  Juerg Schwitter,et al.  ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure 2012 , 2010, European journal of heart failure.

[4]  Gustavo Carneiro,et al.  Left ventricle segmentation from cardiac MRI combining level set methods with deep belief networks , 2013, 2013 IEEE International Conference on Image Processing.

[5]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[6]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[7]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[8]  P. Matthews,et al.  UK Biobank’s cardiovascular magnetic resonance protocol , 2015, Journal of Cardiovascular Magnetic Resonance.

[9]  Trevor Darrell,et al.  Fully convolutional networks for semantic segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  J P Greenwood,et al.  Cardiovascular magnetic resonance imaging: what the general cardiologist should know , 2016, Heart.

[15]  Hamid Jafarkhani,et al.  A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI , 2015, Medical Image Anal..

[16]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[17]  Volkmar Falk,et al.  2016 ESC Guidelines for the Diagnosis and Treatment of Acute and Chronic Heart Failure. , 2016, Revista espanola de cardiologia.

[18]  Phi Vu Tran,et al.  A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI , 2016, ArXiv.

[19]  Stefan K. Piechnik,et al.  Reference ranges for cardiac structure and function using cardiovascular magnetic resonance (CMR) in Caucasians from the UK Biobank population cohort , 2017, Journal of Cardiovascular Magnetic Resonance.

[20]  Gustavo Carneiro,et al.  Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance , 2017, Medical Image Anal..

[21]  Liming Wang,et al.  An artificial intelligence platform for the multihospital collaborative management of congenital cataracts , 2017, Nature Biomedical Engineering.

[22]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[23]  C. Sudlow,et al.  Comparison of Sociodemographic and Health-Related Characteristics of UK Biobank Participants With Those of the General Population , 2017, American journal of epidemiology.

[24]  Gary Marcus,et al.  Deep Learning: A Critical Appraisal , 2018, ArXiv.

[25]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[26]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.