Improving the Generalizability of Convolutional Neural Network-Based Segmentation on CMR Images

Background: Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g., same scanner or site), their performance often degrades dramatically on images from different scanners or clinical sites. Methods: We propose a simple yet effective way for improving the network generalization ability by carefully designing data normalization and augmentation strategies to accommodate common scenarios in multi-site, multi-scanner clinical imaging data sets. We demonstrate that a neural network trained on a single-site single-scanner dataset from the UK Biobank can be successfully applied to segmenting cardiac MR images across different sites and different scanners without substantial loss of accuracy. Specifically, the method was trained on a large set of 3,975 subjects from the UK Biobank. It was then directly tested on 600 different subjects from the UK Biobank for intra-domain testing and two other sets for cross-domain testing: the ACDC dataset (100 subjects, 1 site, 2 scanners) and the BSCMR-AS dataset (599 subjects, 6 sites, 9 scanners). Results: The proposed method produces promising segmentation results on the UK Biobank test set which are comparable to previously reported values in the literature, while also performing well on cross-domain test sets, achieving a mean Dice metric of 0.90 for the left ventricle, 0.81 for the myocardium, and 0.82 for the right ventricle on the ACDC dataset; and 0.89 for the left ventricle, 0.83 for the myocardium on the BSCMR-AS dataset. Conclusions: The proposed method offers a potential solution to improve CNN-based model generalizability for the cross-scanner and cross-site cardiac MR image segmentation task.

[1]  Thoralf Niendorf,et al.  Comprehensive Cardiac Magnetic Resonance Imaging at 3.0 Tesla: Feasibility and Implications for Clinical Applications , 2006, Investigative radiology.

[2]  Phi Vu Tran,et al.  A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI , 2016, ArXiv.

[3]  Dong Ni,et al.  Left Ventricle Segmentation and Quantification with Attention-Enhanced Segmentation and Shape Correction , 2019, ISICDM.

[4]  Juerg Schwitter,et al.  Current artefacts in cardiac and chest magnetic resonance imaging: tips and tricks. , 2016, The British journal of radiology.

[5]  Daniel Rueckert,et al.  Multi-Task Learning for Left Atrial Segmentation on GE-MRI , 2018, STACOM@MICCAI.

[6]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[7]  Caroline Petitjean,et al.  A review of segmentation methods in short axis cardiac MR images , 2011, Medical Image Anal..

[8]  C. Sudlow,et al.  Comparison of Sociodemographic and Health-Related Characteristics of UK Biobank Participants With Those of the General Population , 2017, American journal of epidemiology.

[9]  Ganapathy Krishnamurthi,et al.  Fully convolutional multi‐scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers , 2018, Medical Image Anal..

[10]  Stefan K. Piechnik,et al.  Reference ranges for cardiac structure and function using cardiovascular magnetic resonance (CMR) in Caucasians from the UK Biobank population cohort , 2017, Journal of Cardiovascular Magnetic Resonance.

[11]  Alejandro F. Frangi,et al.  Automatic initialization and quality control of large‐scale cardiac MRI segmentations , 2018, Medical Image Anal..

[12]  Klaus H. Maier-Hein,et al.  Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges , 2017, Lecture Notes in Computer Science.

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Matthew Albert,et al.  Tips and Tricks , 2020, Clinics in Colon and Rectal Surgery.

[15]  Ben Glocker,et al.  Automated cardiovascular magnetic resonance image analysis with fully convolutional networks , 2017, Journal of Cardiovascular Magnetic Resonance.

[16]  P. Matthews,et al.  UK Biobank’s cardiovascular magnetic resonance protocol , 2015, Journal of Cardiovascular Magnetic Resonance.

[17]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[18]  Jiaying Liu,et al.  Adaptive Batch Normalization for practical domain adaptation , 2018, Pattern Recognit..

[19]  Marc Pollefeys,et al.  An Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation , 2017, STACOM@MICCAI.

[20]  Sanjay K. Prasad,et al.  Myocardial Scar and Mortality in Severe Aortic Stenosis , 2018, Circulation.

[21]  S. Dwivedi,et al.  Obesity May Be Bad: Compressed Convolutional Networks for Biomedical Image Segmentation , 2020 .

[22]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[23]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[24]  S. Plein,et al.  Deep Learning-based Method for Fully Automatic Quantification of Left Ventricle Function from Cine MR Images: A Multivendor, Multicenter Study. , 2019, Radiology.

[25]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[26]  Nicholas Ayache,et al.  A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images , 2014, Medical Image Anal..

[27]  Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , 2018, NeurIPS.

[28]  Mitko Veta,et al.  Roto-Translation Covariant Convolutional Networks for Medical Image Analysis , 2018, MICCAI.

[29]  Hugo Larochelle,et al.  Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 , 2015, ICML.

[30]  Avan Suinesiaputra,et al.  Left ventricular shape variation in asymptomatic populations: the multi-ethnic study of atherosclerosis , 2014, Journal of Cardiovascular Magnetic Resonance.

[31]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[32]  Prabhakar Rajiah,et al.  Cardiovascular MR imaging at 3 T: opportunities, challenges, and solutions. , 2014, Radiographics : a review publication of the Radiological Society of North America, Inc.

[33]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[34]  M. Szklo,et al.  The Multi-Ethnic Study of Atherosclerosis (MESA) , 2012 .

[35]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[36]  Nicholas Ayache,et al.  3-D Consistent and Robust Segmentation of Cardiac Images by Deep Learning With Spatial Propagation , 2018, IEEE Transactions on Medical Imaging.

[37]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[38]  Alejandro F. Frangi,et al.  Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 , 2018, Lecture Notes in Computer Science.

[39]  Silvio Savarese,et al.  Generalizing to Unseen Domains via Adversarial Data Augmentation , 2018, NeurIPS.

[40]  Ender Konukoglu,et al.  Semi-Supervised and Task-Driven Data Augmentation , 2019, IPMI.

[41]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[42]  Olivier Bernard,et al.  Cardiac MRI Segmentation with Strong Anatomical Guarantees , 2019, MICCAI.

[43]  Costin-Anton Boiangiu,et al.  A Study of Image Upsampling and Downsampling Filters , 2019, Comput..

[44]  Hervé Delingette,et al.  Learning a Probabilistic Model for Diffeomorphic Registration , 2018, IEEE Transactions on Medical Imaging.

[45]  Neil Maredia,et al.  The incremental impact of cardiac MRI on clinical decision-making. , 2016, The British journal of radiology.

[46]  Ben Glocker,et al.  Automated quality control in image segmentation: application to the UK Biobank cardiovascular magnetic resonance imaging study , 2019, Journal of Cardiovascular Magnetic Resonance.

[47]  Sander Dieleman,et al.  Rotation-invariant convolutional neural networks for galaxy morphology prediction , 2015, ArXiv.

[48]  Jichao Zhao,et al.  Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges , 2018, Lecture Notes in Computer Science.