Privacy Preserving Domain Adaptation for Semantic Segmentation of Medical Images

Convolutional neural networks (CNNs) have led to significant improvements in tasks involving semantic segmentation of images. CNNs are vulnerable in the area of biomedical image segmentation because of distributional gap between two source and target domains with different data modalities which leads to domain shift. Domain shift makes data annotations in new modalities necessary because models must be retrained from scratch. Unsupervised domain adaptation (UDA) is proposed to adapt a model to new modalities using solely unlabeled target domain data. Common UDA algorithms require access to data points in the source domain which may not be feasible in medical imaging due to privacy concerns. In this work, we develop an algorithm for UDA in a privacy-constrained setting, where the source domain data is inaccessible. Our idea is based on encoding the information from the source samples into a prototypical distribution that is used as an intermediate distribution for aligning the target domain distribution with the source domain distribution. We demonstrate the effectiveness of our algorithm by comparing it to state-ofthe-art medical image semantic segmentation approaches on two medical image semantic segmentation datasets.

[1]  Mohammad Rostami,et al.  Generative Continual Concept Learning , 2019, AAAI.

[2]  Pietro Zanuttigh,et al.  Unsupervised Domain Adaptation in Semantic Segmentation: a Review , 2020, ArXiv.

[3]  Mohammad Rostami,et al.  Sequential Unsupervised Domain Adaptation through Prototypical Distributions , 2020, ArXiv.

[4]  R. Venkatesh Babu,et al.  Universal Source-Free Domain Adaptation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Dongyu Zhang,et al.  Few-Shot Structured Domain Adaptation for Virtual-to-Real Scene Parsing , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[6]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[7]  Hao Chen,et al.  Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss , 2018, IJCAI.

[8]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction , 2018, ArXiv.

[9]  Shunxing Bao,et al.  Adversarial synthesis learning enables segmentation without target modality ground truth , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[10]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Ming-Hsuan Yang,et al.  Learning to Adapt Structured Output Space for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Chong-Wah Ngo,et al.  Transferrable Prototypical Networks for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Fan Zhang,et al.  PDAM: A Panoptic-Level Feature Alignment Framework for Unsupervised Domain Adaptive Instance Segmentation in Microscopy Images , 2020, IEEE Transactions on Medical Imaging.

[15]  Changick Kim,et al.  Self-Ensembling With GAN-Based Data Augmentation for Domain Adaptation in Semantic Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Mohammad Rostami,et al.  Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay , 2019, IJCAI.

[17]  Dong Xu,et al.  Collaborative and Adversarial Network for Unsupervised Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Roland Badeau,et al.  Generalized Sliced Wasserstein Distances , 2019, NeurIPS.

[19]  Nassir Navab,et al.  GANs for Medical Image Analysis , 2018, Artif. Intell. Medicine.

[20]  John F. Canny,et al.  Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Cristiano Saltori,et al.  SF-UDA3D: Source-Free Unsupervised Domain Adaptation for LiDAR-Based 3D Object Detection , 2020, 2020 International Conference on 3D Vision (3DV).

[22]  Quinn Jones,et al.  Few-Shot Adversarial Domain Adaptation , 2017, NIPS.

[23]  M. Shah,et al.  Object tracking: A survey , 2006, CSUR.

[24]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[25]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Tianzhu Zhang,et al.  GCAN: Graph Convolutional Adversarial Network for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[28]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[29]  Mohammad Rostami,et al.  A crowdsourcing triage algorithm for geopolitical event forecasting , 2018, RecSys.

[30]  Mohammad Rostami,et al.  Learning Transferable Knowledge Through Embedding Spaces , 2019 .

[31]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Mohammad Rostami,et al.  Unsupervised Model Adaptation for Continual Semantic Segmentation , 2020, AAAI.

[33]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[34]  Shunxing Bao,et al.  SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth , 2018, IEEE Transactions on Medical Imaging.

[35]  Pingkun Yan,et al.  Unsupervised Domain Adaptation with Dual-Scheme Fusion Network for Medical Image Segmentation , 2020, IJCAI.

[36]  Marc Sebban,et al.  Deep multi-Wasserstein unsupervised domain adaptation , 2019, Pattern Recognit. Lett..

[37]  Yinghuan Shi,et al.  Deep Symmetric Adaptation Network for Cross-Modality Medical Image Segmentation , 2021, IEEE Transactions on Medical Imaging.

[38]  Junzhou Huang,et al.  Progressive Feature Alignment for Unsupervised Domain Adaptation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Kate Saenko,et al.  Correlation Alignment for Unsupervised Domain Adaptation , 2016, Domain Adaptation in Computer Vision Applications.

[40]  Luc Van Gool,et al.  End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners , 2018, ECCV.

[41]  Hao Chen,et al.  Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation , 2018, MLMI@MICCAI.

[42]  Hao Chen,et al.  Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[43]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Hao Chen,et al.  Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation , 2019, AAAI.

[45]  C. Villani,et al.  Quantitative Concentration Inequalities for Empirical Measures on Non-compact Spaces , 2005, math/0503123.

[46]  Paul Magron,et al.  Unsupervised Adversarial Domain Adaptation Based on The Wasserstein Distance For Acoustic Scene Classification , 2019, 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA).

[47]  Xiaoming Liu,et al.  Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs , 2017, Artif. Intell. Medicine.

[48]  Xinlei Chen,et al.  Prior-Aware Neural Network for Partially-Supervised Multi-Organ Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[49]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[51]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[52]  Ievgen Redko,et al.  Theoretical Analysis of Domain Adaptation with Optimal Transport , 2016, ECML/PKDD.

[53]  Xiahai Zhuang,et al.  Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI , 2016, Medical Image Anal..

[54]  Cheng Chen,et al.  PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation , 2019, IEEE Access.

[55]  Mohammad Rostami,et al.  Domain Adaptation for Sentiment Analysis Using Increased Intraclass Separation , 2021, ArXiv.

[56]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[57]  Chen-Yu Lee,et al.  Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Linda G. Shapiro,et al.  Estimating Image Segmentation Difficulty , 2011, MLDM.

[59]  Jean-Philippe Thiran,et al.  Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation , 2021, IEEE Transactions on Medical Imaging.

[60]  Yu Zhao,et al.  Knowledge-Aided Convolutional Neural Network for Small Organ Segmentation , 2019, IEEE Journal of Biomedical and Health Informatics.

[61]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[62]  Eric Eaton,et al.  Deep Transfer Learning for Few-Shot SAR Image Classification , 2019, Remote. Sens..

[63]  Hua Yang,et al.  Online Multi-Object Tracking with Dual Matching Attention Networks , 2018, ECCV.

[64]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Yue Zhang,et al.  Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-ray Image Segmentation , 2018, MICCAI.

[66]  Carlos D. Castillo,et al.  Generate to Adapt: Aligning Domains Using Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[67]  Sethuraman Panchanathan,et al.  Deep Hashing Network for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[68]  Li Wang,et al.  Anatomy-Regularized Representation Learning for Cross-Modality Medical Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[69]  Quentin Mérigot,et al.  A Multiscale Approach to Optimal Transport , 2011, Comput. Graph. Forum.

[70]  Luca Bertinetto,et al.  Fully-Convolutional Siamese Networks for Object Tracking , 2016, ECCV Workshops.

[71]  Eric Eaton,et al.  SAR Image Classification Using Few-Shot Cross-Domain Transfer Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).