Disentangle domain features for cross-modality cardiac image segmentation

Unsupervised domain adaptation (UDA) generally learns a mapping to align the distribution of the source domain and target domain. The learned mapping can boost the performance of the model on the target data, of which the labels are unavailable for model training. Previous UDA methods mainly focus on domain-invariant features (DIFs) without considering the domain-specific features (DSFs), which could be used as complementary information to constrain the model. In this work, we propose a new UDA framework for cross-modality image segmentation. The framework first disentangles each domain into the DIFs and DSFs. To enhance the representation of DIFs, self-attention modules are used in the encoder which allows attention-driven, long-range dependency modeling for image generation tasks. Furthermore, a zero loss is minimized to enforce the information of target (source) DSFs, contained in the source (target) images, to be as close to zero as possible. These features are then iteratively decoded and encoded twice to maintain the consistency of the anatomical structure. To improve the quality of the generated images and segmentation results, several discriminators are introduced for adversarial learning. Finally, with the source data and their DIFs, we train a segmentation network, which can be applicable to target images. We validated the proposed framework for cross-modality cardiac segmentation using two public datasets, and the results showed our method delivered promising performance and compared favorably to state-of-the-art approaches in terms of segmentation accuracies. The source code of this work will be released via https://zmiclab.github.io/projects.html, once this manuscript is accepted for publication.

[1]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[2]  Fan Zhang,et al.  Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation , 2019, MICCAI.

[3]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[4]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[5]  Ben Glocker,et al.  Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images , 2018, Medical Image Anal..

[6]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[7]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Xiahai Zhuang,et al.  Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI , 2016, Medical Image Anal..

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Hao Chen,et al.  Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation , 2019, AAAI.

[11]  Ke Lu,et al.  Locality Preserving Joint Transfer for Domain Adaptation , 2019, IEEE Transactions on Image Processing.

[12]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[13]  Xiahai Zhuang,et al.  CF Distance: A New Domain Discrepancy Metric and Application to Explicit Domain Adaptation for Cross-Modality Cardiac Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[14]  Zijian Li,et al.  Learning Disentangled Semantic Representation for Domain Adaptation , 2019, IJCAI.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Lin Yang,et al.  Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  U. Rajendra Acharya,et al.  Application of deep transfer learning for automated brain abnormality classification using MR images , 2019, Cognitive Systems Research.

[18]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Toby P. Breckon,et al.  Unifying Unsupervised Domain Adaptation and Zero-Shot Visual Recognition , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[21]  Stefano Soatto,et al.  Unsupervised Domain Adaptation via Regularized Conditional Alignment , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Faisal Mahmood,et al.  Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training , 2017, IEEE Transactions on Medical Imaging.

[23]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[24]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Jon Kleinberg,et al.  Transfusion: Understanding Transfer Learning for Medical Imaging , 2019, NeurIPS.

[26]  Gabriela Csurka,et al.  Domain Adaptation for Visual Applications: A Comprehensive Survey , 2017, ArXiv.

[27]  Zi Huang,et al.  Cycle-consistent Conditional Adversarial Transfer Networks , 2019, ACM Multimedia.

[28]  Xiahai Zhuang,et al.  Multivariate Mixture Model for Cardiac Segmentation from Multi-Sequence MRI , 2016, MICCAI.

[29]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[30]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Chao Chen,et al.  Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation , 2018, AAAI.

[32]  Mirella Lapata,et al.  Long Short-Term Memory-Networks for Machine Reading , 2016, EMNLP.

[33]  Yaozong Gao,et al.  Dual‐core steered non‐rigid registration for multi‐modal images via bi‐directional image synthesis , 2017, Medical Image Anal..

[34]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[35]  Xiahai Zhuang,et al.  Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Hyunsoo Kim,et al.  Learning to Discover Cross-Domain Relations with Generative Adversarial Networks , 2017, ICML.

[37]  Michael I. Jordan,et al.  Unsupervised Domain Adaptation with Residual Transfer Networks , 2016, NIPS.

[38]  Daniel Rueckert,et al.  Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation , 2019, STACOM@MICCAI.

[39]  Yoshua Bengio,et al.  Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[40]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Chao Chen,et al.  HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptation , 2019, AAAI.

[42]  Dong Yang,et al.  Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation , 2020, Medical Image Anal..

[43]  Guang Yang,et al.  Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge , 2019, Medical Image Anal..

[44]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[45]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[46]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Lior Wolf,et al.  Domain Intersection and Domain Difference , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[48]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[49]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Yu-Ding Lu,et al.  DRIT++: Diverse Image-to-Image Translation via Disentangled Representations , 2020, International Journal of Computer Vision.

[51]  Yang Zou,et al.  Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training , 2018, ArXiv.

[52]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.