Privacy-preserving Federated Adversarial Domain Adaption over Feature Groups for Interpretability

We present a novel privacy-preserving federated adversarial domain adaptation approach (PrADA) to address an under-studied but practical cross-silo federated domain adaptation problem, in which the party of the target domain is insufficient in both samples and features. We address the lackof-feature issue by extending the feature space through vertical federated learning with a feature-rich party and tackle the sample-scarce issue by performing adversarial domain adaptation from the sample-rich source party to the target party. In this work, we focus on financial applications where interpretability is critical. However, existing adversarial domain adaptation methods typically apply a single feature extractor to learn feature representations that are low-interpretable with respect to the target task. To improve interpretability, we exploit domain expertise to split the feature space into multiple groups that each holds relevant features, and we learn a semantically meaningful high-order feature from each feature group. In addition, we apply a feature extractor (along with a domain discriminator) for each feature group to enable a fine-grained domain adaptation. We design a secure protocol that enables performing the PrADA in a secure and efficient manner. We evaluate our approach on two tabular datasets. Experiments demonstrate both the effectiveness and practicality of our approach.

[1]  Mehryar Mohri,et al.  Agnostic Federated Learning , 2019, ICML.

[2]  Yifei Zhang,et al.  Additively Homomorphical Encryption based Deep Neural Network for Asymmetrically Collaborative Machine Learning , 2020, ArXiv.

[3]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[4]  Kate Saenko,et al.  Federated Adversarial Domain Adaptation , 2020, ICLR.

[5]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Ercan E. Kuruoglu 30th anniversary of Digital Signal Processing: a Review Journal , 2021, Digit. Signal Process..

[7]  Cynthia Rudin,et al.  This Looks Like That: Deep Learning for Interpretable Image Recognition , 2018 .

[8]  Yun Zhang,et al.  Privacy-Preserving Unsupervised Domain Adaptation in Federated Setting , 2020, IEEE Access.

[9]  Sarah Adel Bargal,et al.  NBDT: Neural-Backed Decision Trees , 2020, ArXiv.

[10]  Lucy Rosenbloom arXiv , 2019, The Charleston Advisor.

[11]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[12]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Pascal Paillier,et al.  Public-Key Cryptosystems Based on Composite Degree Residuosity Classes , 1999, EUROCRYPT.

[14]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[15]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[16]  J. Eubanks,et al.  Fate , 2010, Annals of Internal Medicine.

[17]  A. Gabriel Editor , 2018, Best "New" African Poets 2018 Anthology.

[18]  Tommi S. Jaakkola,et al.  Towards Robust Interpretability with Self-Explaining Neural Networks , 2018, NeurIPS.

[19]  B. P. Murphy,et al.  Handbook of Methods of Applied Statistics , 1968 .

[20]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Nicha C. Dvornek,et al.  Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Results , 2020, Medical Image Anal..

[22]  Virendra J. Marathe,et al.  Private Federated Learning with Domain Adaptation , 2019, ArXiv.

[23]  Jaime G. Carbonell,et al.  Characterizing and Avoiding Negative Transfer , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[25]  Cong Wang,et al.  GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning , 2018, IJCAI.