Simultaneous left atrium anatomy and scar segmentations via deep learning in multiview information with attention

Three-dimensional late gadolinium enhanced (LGE) cardiac MR (CMR) of left atrial scar in patients with atrial fibrillation (AF) has recently emerged as a promising technique to stratify patients, to guide ablation therapy and to predict treatment success. This requires a segmentation of the high intensity scar tissue and also a segmentation of the left atrium (LA) anatomy, the latter usually being derived from a separate bright-blood acquisition. Performing both segmentations automatically from a single 3D LGE CMR acquisition would eliminate the need for an additional acquisition and avoid subsequent registration issues. In this paper, we propose a joint segmentation method based on multiview two-task (MVTT) recursive attention model working directly on 3D LGE CMR images to segment the LA (and proximal pulmonary veins) and to delineate the scar on the same dataset. Using our MVTT recursive attention model, both the LA anatomy and scar can be segmented accurately (mean Dice score of 93% for the LA anatomy and 87% for the scar segmentations) and efficiently (∼0.27 s to simultaneously segment the LA anatomy and scars directly from the 3D LGE CMR dataset with 60–68 2D slices). Compared to conventional unsupervised learning and other state-of-the-art deep learning based methods, the proposed MVTT model achieved excellent results, leading to an automatic generation of a patient-specific anatomical model combined with scar segmentation for patients in AF.

[1]  Guang Yang,et al.  Bayesian VoxDRN: A Probabilistic Deep Voxelwise Dilated Residual Network for Whole Heart Segmentation from 3D MR Images , 2018, MICCAI.

[2]  D. Firmin,et al.  Navigator artifact reduction in three‐dimensional late gadolinium enhancement imaging of the atria , 2014, Magnetic resonance in medicine.

[3]  Heye Zhang,et al.  Segmentation and quantification of infarction without contrast agents via spatiotemporal generative adversarial learning , 2020, Medical Image Anal..

[4]  Lluís Mont,et al.  CMR-guided approach to localize and ablate gaps in repeat AF ablation procedure. , 2014, JACC. Cardiovascular imaging.

[5]  Kawal S. Rhode,et al.  A Novel Skeleton Based Quantification and 3-D Volumetric Visualization of Left Atrium Fibrosis Using Late Gadolinium Enhancement Magnetic Resonance Imaging , 2014, IEEE Transactions on Medical Imaging.

[6]  Yiyi Zhang,et al.  Left Atrial LGE and Arrhythmia Recurrence Following Pulmonary Vein Isolation for Paroxysmal and Persistent AF. , 2016, JACC. Cardiovascular imaging.

[7]  R. J. van der Geest,et al.  Fully automatic segmentation of left atrium and pulmonary veins in late gadolinium‐enhanced MRI: Towards objective atrial scar assessment , 2016, Journal of magnetic resonance imaging : JMRI.

[8]  Robert S. MacLeod,et al.  Automatic classification of scar tissue in late gadolinium enhancement cardiac MRI for the assessment of left-atrial wall injury after radiofrequency ablation , 2012, Medical Imaging.

[9]  E. Kholmovski,et al.  Atrial Fibrosis Helps Select the Appropriate Patient and Strategy in Catheter Ablation of Atrial Fibrillation: A DE‐MRI Guided Approach , 2011, Journal of cardiovascular electrophysiology.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Jennifer Keegan,et al.  Improved respiratory efficiency of 3D late gadolinium enhancement imaging using the continuously adaptive windowing strategy (CLAWS) , 2014, Magnetic resonance in medicine.

[12]  José Angel Cabrera,et al.  Anatomic Relations Between the Esophagus and Left Atrium and Relevance for Ablation of Atrial Fibrillation , 2005, Circulation.

[13]  Reza Nezafat,et al.  Recurrence of atrial fibrillation correlates with the extent of post-procedural late gadolinium enhancement: a pilot study. , 2009, JACC. Cardiovascular imaging.

[14]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[15]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[16]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[17]  Jennifer Keegan,et al.  Dynamic inversion time for improved 3D late gadolinium enhancement imaging in patients with atrial fibrillation , 2015, Magnetic resonance in medicine.

[18]  Christopher McGann,et al.  Atrial Fibrillation Ablation Outcome Is Predicted by Left Atrial Remodeling on MRI , 2014, Circulation. Arrhythmia and electrophysiology.

[19]  Kun Zhang,et al.  Multi-scale Masked 3-D U-Net for Brain Tumor Segmentation , 2018, BrainLes@MICCAI.

[20]  Jichao Zhao,et al.  Fully Automatic Left Atrium Segmentation From Late Gadolinium Enhanced Magnetic Resonance Imaging Using a Dual Fully Convolutional Neural Network , 2019, IEEE Transactions on Medical Imaging.

[21]  Kawal S. Rhode,et al.  Repeat Left Atrial Catheter Ablation: Cardiac Magnetic Resonance Prediction of Endocardial Voltage and Gaps in Ablation Lesion Sets , 2015, Circulation. Arrhythmia and electrophysiology.

[22]  Kawal S. Rhode,et al.  CardiacNET: Segmentation of Left Atrium and Proximal Pulmonary Veins from MRI Using Multi-view CNN , 2017, MICCAI.

[23]  Joshua J. E. Blauer,et al.  Detection and Quantification of Left Atrial Structural Remodeling With Delayed-Enhancement Magnetic Resonance Imaging in Patients With Atrial Fibrillation , 2009, Circulation.

[24]  Rob S MacLeod,et al.  Evaluation of Left Atrial Lesions After Initial and Repeat Atrial Fibrillation Ablation: Lessons Learned From Delayed-Enhancement MRI in Repeat Ablation Procedures , 2010, Circulation. Arrhythmia and electrophysiology.

[25]  Andreas K. Maier,et al.  Dilated Convolutions in Neural Networks for Left Atrial Segmentation in 3D Gadolinium Enhanced-MRI , 2018, STACOM@MICCAI.

[26]  Alistair A. Young,et al.  Statistical Atlases and Computational Models of the Heart. Imaging and Modelling Challenges , 2013, Lecture Notes in Computer Science.

[27]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[28]  Guang Yang,et al.  Multi-atlas propagation based left atrium segmentation coupled with super-voxel based pulmonary veins delineation in late gadolinium-enhanced cardiac MRI , 2017, Medical Imaging.

[29]  Guang Yang,et al.  Multiview Two-Task Recursive Attention Model for Left Atrium and Atrial Scars Segmentation , 2018, MICCAI.

[30]  Jichao Zhao,et al.  Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges , 2018, Lecture Notes in Computer Science.

[31]  Guang Yang,et al.  Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge , 2019, Medical Image Anal..

[32]  Daniel J. Perry,et al.  Evaluation of current algorithms for segmentation of scar tissue from late Gadolinium enhancement cardiovascular magnetic resonance of the left atrium: an open-access grand challenge , 2013, Journal of Cardiovascular Magnetic Resonance.

[33]  Guang Yang,et al.  A two-stage 3D Unet framework for multi-class segmentation on full resolution image , 2018, ArXiv.

[34]  Guang Yang,et al.  A fully automatic deep learning method for atrial scarring segmentation from late gadolinium-enhanced MRI images , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[35]  Scott D Flamm,et al.  Clinical utility of multimodality LA imaging: assessment of size, function, and structure. , 2011, JACC. Cardiovascular imaging.

[36]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[37]  Guang Yang,et al.  Fully automatic segmentation and objective assessment of atrial scars for long‐standing persistent atrial fibrillation patients using late gadolinium‐enhanced MRI , 2017, Medical physics.

[38]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Bertil Schmidt,et al.  ν-net: Deep Learning for Generalized Biventricular Mass and Function Parameters Using Multicenter Cardiac MRI Data. , 2018, JACC. Cardiovascular imaging.

[40]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[41]  Huafeng Liu,et al.  Direct delineation of myocardial infarction without contrast agents using a joint motion feature learning architecture , 2018, Medical Image Anal..

[43]  Xiahai Zhuang,et al.  Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI , 2016, Medical Image Anal..

[44]  Joshua J. E. Blauer,et al.  New magnetic resonance imaging based method to define extent of left atrial wall injury after the ablation of atrial fibrillation , 2008 .

[45]  P. Platonov,et al.  Left Atrial Posterior Wall Thickness in Patients with and without Atrial Fibrillation: Data from 298 Consecutive Autopsies , 2008, Journal of cardiovascular electrophysiology.