Landmark Detection in Cardiac MRI by Using a Convolutional Neural Network

Purpose To develop a convolutional neural network (CNN) solution for landmark detection in cardiac MRI (CMR). Materials and Methods This retrospective study included cine, late gadolinium enhancement (LGE), and T1 mapping examinations from two hospitals. The training set included 2329 patients (34 089 images; mean age, 54.1 years; 1471 men; December 2017 to March 2020). A hold-out test set included 531 patients (7723 images; mean age, 51.5 years; 323 men; May 2020 to July 2020). CNN models were developed to detect two mitral valve plane and apical points on long-axis images. On short-axis images, anterior and posterior right ventricular (RV) insertion points and left ventricular (LV) center points were detected. Model outputs were compared with manual labels assigned by two readers. The trained model was deployed to MRI scanners. Results For the long-axis images, successful detection of cardiac landmarks ranged from 99.7% to 100% for cine images and from 99.2% to 99.5% for LGE images. For the short-axis images, detection rates were 96.6% for cine, 97.6% for LGE, and 98.7% for T1 mapping. The Euclidean distances between model-assigned and manually assigned labels ranged from 2 to 3.5 mm for different landmarks, indicating close agreement between model-derived landmarks and manually assigned labels. For all views and imaging sequences, no differences between the models’ assessment of images and the readers’ assessment of images were found for the anterior RV insertion angle or LV length. Model inference for a typical cardiac cine series took 610 msec with the graphics processing unit and 5.6 seconds with central processing unit. Conclusion A CNN was developed for landmark detection on both long- and short-axis CMR images acquired with cine, LGE, and T1 mapping sequences, and the accuracy of the CNN was comparable with the interreader variation. Keywords: Cardiac, Heart, Convolutional Neural Network (CNN), Deep Learning Algorithms, Machine Learning Algorithms, Feature Detection, Quantification, Supervised Learning, MR Imaging Supplemental material is available for this article. Published under a CC BY 4.0 license.

[1]  D. Rueckert,et al.  A Multicenter, Scan-Rescan, Human and Machine Learning CMR Study to Test Generalizability and Precision in Imaging Biomarker Analysis. , 2019, Circulation. Cardiovascular imaging.

[2]  Jonathan Tompson,et al.  Efficient object localization using Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Hui Xue,et al.  Automated detection of left ventricle in arterial input function images for inline perfusion mapping using deep learning: A study of 15,000 patients , 2019, Magnetic resonance in medicine.

[4]  Euan A. Ashley,et al.  Weakly supervised classification of aortic valve malformations using unlabeled cardiac MRI sequences , 2019, Nature Communications.

[5]  P. Kellman,et al.  Automated Inline Analysis of Myocardial Perfusion MRI with Deep Learning , 2019, Radiology. Artificial intelligence.

[6]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[7]  Lionel M. Ni,et al.  Generalizing from a Few Examples , 2020, ACM Comput. Surv..

[8]  S. Plein,et al.  Myocardial T1 mapping: Application to patients with acute and chronic myocardial infarction , 2007, Magnetic resonance in medicine.

[9]  Alexander Rakhlin,et al.  Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning , 2018, bioRxiv.

[10]  Shruti Jadon A survey of loss functions for semantic segmentation , 2020, 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB).

[11]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[12]  Christine H. Lorenz,et al.  Unsupervised Inline Analysis of Cardiac Perfusion MRI , 2009, MICCAI.

[13]  Marie-Pierre Jolly,et al.  Motion correction for myocardial T1 mapping using image registration with synthetic image estimation , 2012, Magnetic resonance in medicine.

[14]  Tal Hassner,et al.  Face recognition in unconstrained videos with matched background similarity , 2011, CVPR 2011.

[15]  J. Jögi,et al.  Longitudinal shortening remains the principal component of left ventricular pumping in patients with chronic myocardial infarction even when the absolute atrioventricular plane displacement is decreased , 2017, BMC Cardiovascular Disorders.

[16]  E. Nagel,et al.  Standardized image interpretation and post-processing in cardiovascular magnetic resonance - 2020 update , 2020, Journal of Cardiovascular Magnetic Resonance.

[17]  Artus Krohn-Grimberghe,et al.  Facial Key Points Detection using Deep Convolutional Neural Network - NaimishNet , 2017, ArXiv.

[18]  O. Simonetti,et al.  T2 quantification for improved detection of myocardial edema , 2009, Journal of cardiovascular magnetic resonance : official journal of the Society for Cardiovascular Magnetic Resonance.

[19]  E. McVeigh,et al.  Phase‐sensitive inversion recovery for detecting myocardial infarction using gadolinium‐delayed hyperenhancement † , 2002, Magnetic resonance in medicine.

[20]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[21]  Reza Nezafat,et al.  Three-dimensional Deep Convolutional Neural Networks for Automated Myocardial Scar Quantification in Hypertrophic Cardiomyopathy: A Multicenter Multivendor Study. , 2019, Radiology.

[22]  Peter Kellman,et al.  Dynamic autocalibrated parallel imaging using temporal GRAPPA (TGRAPPA) , 2005, Magnetic resonance in medicine.

[23]  P. Kellman,et al.  Females have higher myocardial perfusion, blood volume and extracellular volume compared to males – an adenosine stress cardiovascular magnetic resonance study , 2020, Scientific Reports.

[24]  James C Moon,et al.  Automated detection of left ventricle in arterial input function images for inline perfusion mapping using deep learning: A study of 15,000 patients , 2020, Magnetic resonance in medicine.

[25]  Mitko Veta,et al.  Automatic cardiac landmark localization by a recurrent neural network , 2019, Medical Imaging: Image Processing.

[26]  Marie-Pierre Jolly,et al.  Phase‐sensitive inversion recovery for myocardial T1 mapping with motion correction and parametric fitting , 2013, Magnetic resonance in medicine.

[27]  P. Kellman,et al.  Free-Breathing, Motion-Corrected Late Gadolinium Enhancement Is Robust and Extends Risk Stratification to Vulnerable Patients , 2013, Circulation. Cardiovascular imaging.

[28]  Tim Leiner,et al.  CNN-based Landmark Detection in Cardiac CTA Scans , 2018, ArXiv.

[29]  P. Kellman,et al.  Myocardial Fat Imaging , 2010, Current cardiovascular imaging reports.

[30]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[31]  Andrew C Larson,et al.  Motion‐corrected free‐breathing delayed enhancement imaging of myocardial infarction , 2005, Magnetic resonance in medicine.

[32]  Scott D Flamm,et al.  Standardized image interpretation and post processing in cardiovascular magnetic resonance: Society for Cardiovascular Magnetic Resonance (SCMR) Board of Trustees Task Force on Standardized Post Processing , 2013, Journal of Cardiovascular Magnetic Resonance.

[33]  P. Kellman,et al.  T1-mapping in the heart: accuracy and precision , 2014, Journal of Cardiovascular Magnetic Resonance.

[34]  S. Plein,et al.  Deep Learning-based Method for Fully Automatic Quantification of Left Ventricle Function from Cine MR Images: A Multivendor, Multicenter Study. , 2019, Radiology.

[35]  Andrew Zisserman,et al.  Recurrent Human Pose Estimation , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[36]  Evan M Masutani,et al.  Deep Learning-based Prescription of Cardiac MRI Planes. , 2019, Radiology. Artificial intelligence.

[37]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[38]  Sonia Nielles-Vallespin,et al.  Myocardial perfusion cardiovascular magnetic resonance: optimized dual sequence and reconstruction for quantification , 2017, Journal of Cardiovascular Magnetic Resonance.

[39]  Dong Seog Han,et al.  Facial Keypoint Detection with Convolutional Neural Networks , 2020, 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[40]  Ben Glocker,et al.  Automated cardiovascular magnetic resonance image analysis with fully convolutional networks , 2017, Journal of Cardiovascular Magnetic Resonance.

[41]  Yuanyuan Wang,et al.  MRI Manufacturer Shift and Adaptation: Increasing the Generalizability of Deep Learning Segmentation for MR Images Acquired with Different Scanners. , 2020, Radiology. Artificial intelligence.

[42]  Victor Mor-Avi,et al.  Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. , 2015, European heart journal cardiovascular Imaging.

[43]  Andrew Zisserman,et al.  Flowing ConvNets for Human Pose Estimation in Videos , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).