FedCKE: Cross-Domain Knowledge Graph Embedding in Federated Learning

Representing the structural relations between entities, i.e., knowledge graph embedding, which is a method to learn low-dimensional representations of knowledge, has become an increasingly prevalent research orientation in cognitive and human intelligence. It is significant to study how to interrelate, fuse and embed the knowledge graph data from different domains while considering data not shared. In this paper, we propose a model of cross-domain knowledge graph embedding in federated learning (FedCKE), in which entity/relation embedding between different domains can interact securely in the case that data is not shared. In advance of client model training, we present an inter-domain encrypted entity/relation alignment method using the encrypted sample alignment method in vertical federated learning, which can obtain entity/relation intersections between different domains without revealing any triples structure and additional entities/relations in the respective datasets. On the server, we aggregate the same entity/relation embeddings by the association in conjunction with the parameter-secure aggregation method in horizontal federated learning. Experimental results on three real datasets show that the proposed FedCKE model is able to enhance the embedding of different clients (domains).

[1]  Tianrui Li,et al.  Cross-Domain Knowledge Graph Chiasmal Embedding for Multi-Domain Item-Item Recommendation , 2023, IEEE Transactions on Knowledge and Data Engineering.

[2]  A. Mayoue,et al.  A Secure Federated Learning framework using Homomorphic Encryption and Verifiable Computing , 2021, 2021 Reconciling Data Analytics, Automation, Privacy, and Security: A Big Data Challenge (RDAAPS).

[3]  Vincent Wenchen Zheng,et al.  Differentially Private Federated Knowledge Graphs Embedding , 2021, CIKM.

[4]  Shaojie Tang,et al.  Toward Understanding the Influence of Individual Clients in Federated Learning , 2020, AAAI.

[5]  Diego F. Aranha,et al.  Faster unbalanced Private Set Intersection in the semi-honest setting , 2020, J. Cryptogr. Eng..

[6]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2019, Found. Trends Mach. Learn..

[7]  Phillipp Schoppmann,et al.  Asymmetric Private Set Intersection with Applications to Contact Tracing and Private Vertical Federated Machine Learning , 2020, ArXiv.

[8]  Tianjian Chen,et al.  A Secure Federated Transfer Learning Framework , 2020, IEEE Intelligent Systems.

[9]  Wenqi Wei,et al.  LDP-Fed: federated learning with local differential privacy , 2020, EdgeSys@EuroSys.

[10]  Yang Liu,et al.  Asymmetrical Vertical Federated Learning , 2020, ArXiv.

[11]  Tianjian Chen,et al.  A Fairness-aware Incentive Scheme for Federated Learning , 2020, AIES.

[12]  H. Vincent Poor,et al.  Federated Learning With Differential Privacy: Algorithms and Performance Analysis , 2019, IEEE Transactions on Information Forensics and Security.

[13]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[14]  Yang Liu,et al.  Federated Learning , 2019, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[15]  Yuanming Shi,et al.  A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression , 2019, ArXiv.

[16]  Junpu Wang,et al.  FedMD: Heterogenous Federated Learning via Model Distillation , 2019, ArXiv.

[17]  Ying-Chang Liang,et al.  Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach , 2019, 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS).

[18]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[19]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2018, IEEE Transactions on Information Forensics and Security.

[20]  Richard Nock,et al.  Entity Resolution and Federated Learning get a Federated Resolution , 2018, ArXiv.

[21]  H. Brendan McMahan,et al.  Learning Differentially Private Recurrent Language Models , 2017, ICLR.

[22]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[23]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[24]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[25]  Bin Liang,et al.  CN-DBpedia: A Never-Ending Chinese Knowledge Extraction System , 2017, IEA/AIE.

[26]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[27]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[28]  Zhifang Sui,et al.  Encoding Temporal Information for Time-Aware Link Prediction , 2016, EMNLP.

[29]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[30]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[31]  Jun Zhao,et al.  Learning to Represent Knowledge Graphs with Gaussian Embedding , 2015, CIKM.

[32]  Benny Pinkas,et al.  Phasing: Private Set Intersection Using Permutation-based Hashing , 2015, USENIX Security Symposium.

[33]  Danqi Chen,et al.  Observed versus latent features for knowledge base and text inference , 2015, CVSC.

[34]  Bo Yang,et al.  Private set intersection via public key encryption with keywords search , 2015, Secur. Commun. Networks.

[35]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[36]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[37]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[38]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[39]  Yizhou Sun,et al.  Personalized entity recommendation: a heterogeneous information network approach , 2014, WSDM.

[40]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[41]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[42]  Antoine Bordes,et al.  A semantic matching energy function for learning with multi-relational data , 2013, Machine Learning.

[43]  Jason Weston,et al.  A semantic matching energy function for learning with multi-relational data , 2013, Machine Learning.

[44]  Nicolas Le Roux,et al.  A latent factor model for highly multi-relational data , 2012, NIPS.

[45]  Hans-Peter Kriegel,et al.  Factorizing YAGO: scalable machine learning for linked data , 2012, WWW.

[46]  Emiliano De Cristofaro,et al.  On the performance of certain Private Set Intersection protocols. (And some remarks on the recent paper by Huang et al. in NDSS'12) , 2012, IACR Cryptol. ePrint Arch..

[47]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[48]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[49]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[50]  Emiliano De Cristofaro,et al.  Practical Private Set Intersection Protocols with Linear Complexity , 2010, Financial Cryptography.

[51]  Emiliano De Cristofaro,et al.  Practical Private Set Intersection Protocols with Linear Computational and Bandwidth Complexity , 2009, IACR Cryptol. ePrint Arch..

[52]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[53]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[54]  Benny Pinkas,et al.  Efficient Private Matching and Set Intersection , 2004, EUROCRYPT.

[55]  Tad Hogg,et al.  Enhancing privacy and trust in electronic communities , 1999, EC '99.