A Survey on federated learning*

Federated learning (FL) is an emerging setting which implement machine learning in a distributed environment while protecting privacy. Research activities relating to FLhave grown at a fast rate recently in control. Exactly what activities have been carrying the research momentum forward is a question of interest to the research community. This study finds these research activities and optimization path of FL based on survey. Thus, this study aims to review related studies of FL to base on the baseline a universal definition gives a guiding for the future work. Besides, this study presents the prevailing FL applications and the evolution of federated learning. In the end, this study also identifies four research fronts to enrich the FL literature and help advance our understanding of the field. A comprehensive taxonomy of FL can also be developed through analyzing the results of this review.

[1]  Anit Kumar Sahu,et al.  Federated Optimization in Heterogeneous Networks , 2018, MLSys.

[2]  Eryk Dutkiewicz,et al.  Energy Demand Prediction with Federated Learning for Electric Vehicle Networks , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[3]  Paul M. Thompson,et al.  Federated Learning in Distributed Medical Databases: Meta-Analysis of Large-Scale Subcortical Brain Data , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[4]  Satoshi Nakamoto.  Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[5]  Kan Yang,et al.  VerifyNet: Secure and Verifiable Federated Learning , 2020, IEEE Transactions on Information Forensics and Security.

[6]  Choong Seon Hong,et al.  FLchain: Federated Learning via MEC-enabled Blockchain Network , 2019, 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS).

[7]  Xiaoyan Sun,et al.  Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Takayuki Nishio,et al.  Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[9]  Wei Shi,et al.  Federated learning of predictive models from federated Electronic Health Records , 2018, Int. J. Medical Informatics.

[10]  Ying-Chang Liang,et al.  Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach , 2019, 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS).

[11]  Qiang Yang,et al.  Cross-domain sentiment classification via spectral feature alignment , 2010, WWW '10.

[12]  Li Huang,et al.  Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records , 2019, J. Biomed. Informatics.

[13]  Rui Zhang,et al.  A Hybrid Approach to Privacy-Preserving Federated Learning , 2018, Informatik Spektrum.

[14]  Swaroop Ramaswamy,et al.  Federated Learning for Emoji Prediction in a Mobile Keyboard , 2019, ArXiv.

[15]  Hongyu Li,et al.  An End-to-End Encrypted Neural Network for Gradient Updates Transmission in Federated Learning , 2019, 2019 Data Compression Conference (DCC).

[16]  Xin Qin,et al.  FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare , 2019, IEEE Intelligent Systems.

[17]  Long Hu,et al.  Privacy-aware service placement for mobile edge computing via federated learning , 2019, Inf. Sci..

[18]  Joseph Dureau,et al.  Federated Learning for Keyword Spotting , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Xiaoyan Sun,et al.  Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Lifeng Sun,et al.  Two-Stream Federated Learning: Reduce the Communication Costs , 2018, 2018 IEEE Visual Communications and Image Processing (VCIP).

[21]  Georgios Spathoulas,et al.  Machine Learning for All: A More Robust Federated Learning Framework , 2019, ICISSP.

[22]  Kin K. Leung,et al.  Adaptive Federated Learning in Resource Constrained Edge Computing Systems , 2018, IEEE Journal on Selected Areas in Communications.

[23]  Vitaly Shmatikov,et al.  How To Backdoor Federated Learning , 2018, AISTATS.

[24]  Bing Chen,et al.  Poisoning Attack in Federated Learning using Generative Adversarial Nets , 2019, 2019 18th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/13th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE).

[25]  Tianjian Chen,et al.  Abnormal Client Behavior Detection in Federated Learning , 2019, ArXiv.

[26]  Walid Saad,et al.  Federated Echo State Learning for Minimizing Breaks in Presence in Wireless Virtual Reality Networks , 2018, IEEE Transactions on Wireless Communications.

[27]  Prateek Mittal,et al.  Analyzing Federated Learning through an Adversarial Lens , 2018, ICML.

[28]  Albert Y. Zomaya,et al.  Federated Learning over Wireless Networks: Optimization Model Design and Analysis , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[29]  Richard Nock,et al.  Entity Resolution and Federated Learning get a Federated Resolution , 2018, ArXiv.

[30]  Tom Ouyang,et al.  Federated Learning Of Out-Of-Vocabulary Words , 2019, ArXiv.

[31]  Choong Seon Hong,et al.  Blockchain-based Node-aware Dynamic Weighting Methods for Improving Federated Learning Performance , 2019, 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS).

[32]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[33]  Tianjian Chen,et al.  A Secure Federated Transfer Learning Framework , 2020, IEEE Intelligent Systems.

[34]  Yunus Sarikaya,et al.  Motivating Workers in Federated Learning: A Stackelberg Game Perspective , 2019, IEEE Networking Letters.

[35]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[36]  Amir Houmansadr,et al.  Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[37]  Runhua Xu,et al.  HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning , 2019, AISec@CCS.

[38]  Tassilo Klein,et al.  Differentially Private Federated Learning: A Client Level Perspective , 2017, ArXiv.

[39]  Yan Zhang,et al.  Differentially Private Asynchronous Federated Learning for Mobile Edge Computing in Urban Informatics , 2020, IEEE Transactions on Industrial Informatics.

[40]  Yang Song,et al.  Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning , 2018, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[41]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[42]  Jiang Du,et al.  Electronic medical record security sharing model based on blockchain , 2019, ICCSP.

[43]  Dmitriy Dligach,et al.  Two-stage Federated Phenotyping and Patient Representation Learning , 2019, BioNLP@ACL.

[44]  Stephen A. Jarvis,et al.  SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead , 2019, IEEE Transactions on Computers.

[45]  Haomiao Yang,et al.  Towards Efficient and Privacy-Preserving Federated Deep Learning , 2019, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[46]  Haoran Yu,et al.  Visual Inspection with Federated Learning , 2019, ICIAR.

[47]  Seunghak Lee,et al.  More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server , 2013, NIPS.

[48]  Huadong Ma,et al.  Federated Region-Learning: An Edge Computing Based Framework for Urban Environment Sensing , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[49]  Mehryar Mohri,et al.  Agnostic Federated Learning , 2019, ICML.

[50]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[51]  Yi Zhou,et al.  Towards Taming the Resource and Data Heterogeneity in Federated Learning , 2019, OpML.

[52]  Huzefa Rangwala,et al.  Asynchronous Online Federated Learning for Edge Devices , 2019, ArXiv.

[53]  Kejiang Ye,et al.  FFD: A Federated Learning Based Method for Credit Card Fraud Detection , 2019, BigData.

[54]  Sébastien Gambs,et al.  IOTFLA : A Secured and Privacy-Preserving Smart Home Architecture Implementing Federated Learning , 2019, 2019 IEEE Security and Privacy Workshops (SPW).

[55]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[56]  Yaochu Jin,et al.  Multi-Objective Evolutionary Federated Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[57]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2019, Found. Trends Mach. Learn..

[58]  Mariana Raykova,et al.  Privacy-Preserving Distributed Linear Regression on High-Dimensional Data , 2017, Proc. Priv. Enhancing Technol..

[59]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[60]  Kenneth T. Co,et al.  Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging , 2019, ArXiv.

[61]  Yasaman Khazaeni,et al.  Bayesian Nonparametric Federated Learning of Neural Networks , 2019, ICML.

[62]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[63]  Zhu Han,et al.  Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanism , 2019, IEEE Communications Magazine.

[64]  Michael Naehrig,et al.  Private Predictive Analysis on Encrypted Medical Data , 2014, IACR Cryptol. ePrint Arch..

[65]  Yang Liu,et al.  Secure Federated Transfer Learning , 2018, ArXiv.

[66]  Tianjian Chen,et al.  HHHFL: Hierarchical Heterogeneous Horizontal Federated Learning for Electroencephalography , 2019, ArXiv.