Federated Learning on Non-IID Data: A Survey

Federated learning is an emerging distributed machine learning framework for privacy preservation. However, models trained in federated learning usually have worse performance than those trained in the standard centralized learning mode, especially when the training data are not independent and identically distributed (Non-IID) on the local devices. In this survey, we provide a detailed analysis of the influence of Non-IID data on both parametric and non-parametric machine learning models in both horizontal and vertical federated learning. In addition, current research work on handling challenges of Non-IID data in federated learning are reviewed, and both advantages and disadvantages of these approaches are discussed. Finally, we suggest several future research directions before concluding the paper.

[1]  Alexander Binder,et al.  Learning and Evaluation in Presence of Non-i.i.d. Label Noise , 2014, AISTATS.

[2]  Christopher Briggs,et al.  A Review of Privacy Preserving Federated Learning for Private IoT Analytics , 2020, ArXiv.

[3]  Paolo Favaro,et al.  Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles , 2016, ECCV.

[4]  Eric Lin,et al.  FedFMC: Sequential Efficient Federated Learning on Non-iid Data , 2020, ArXiv.

[5]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[6]  Yasaman Khazaeni,et al.  Bayesian Nonparametric Federated Learning of Neural Networks , 2019, ICML.

[7]  Runhua Xu,et al.  HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning , 2019, AISec@CCS.

[8]  Fuad E. Alsaadi,et al.  A competitive mechanism integrated multi-objective whale optimization algorithm with differential evolution , 2021, Neurocomputing.

[9]  Dawn Song,et al.  Model-Agnostic Round-Optimal Federated Learning via Knowledge Transfer , 2020, ArXiv.

[10]  Wojciech Samek,et al.  Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Janis Keuper,et al.  Distributed Training of Deep Neural Networks: Theoretical and Practical Limits of Parallel Scalability , 2016, 2016 2nd Workshop on Machine Learning in HPC Environments (MLHPC).

[12]  Adi Shamir,et al.  How to share a secret , 1979, CACM.

[13]  Sashank J. Reddi,et al.  SCAFFOLD: Stochastic Controlled Averaging for Federated Learning , 2019, ICML.

[14]  Siwei Feng,et al.  Multi-Participant Multi-Class Vertical Federated Learning , 2020, ArXiv.

[15]  Eunho Yang,et al.  FedMix: Approximation of Mixup under Mean Augmented Federated Learning , 2021, ICLR.

[16]  Aditya G. Parameswaran,et al.  Challenges in Data Crowdsourcing , 2016, IEEE Transactions on Knowledge and Data Engineering.

[17]  Qinghua Liu,et al.  Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization , 2020, NeurIPS.

[18]  Ying-Chang Liang,et al.  Federated Learning in Mobile Edge Networks: A Comprehensive Survey , 2020, IEEE Communications Surveys & Tutorials.

[19]  Lingjuan Lyu,et al.  Threats to Federated Learning , 2020, Federated Learning.

[20]  Xiaoyan Sun,et al.  Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Yanzhao Wu,et al.  A Framework for Evaluating Gradient Leakage Attacks in Federated Learning , 2020, ArXiv.

[22]  Yuan Gao,et al.  A survey on federated learning , 2021, Knowl. Based Syst..

[23]  Marc'Aurelio Ranzato,et al.  Multi-GPU Training of ConvNets , 2013, ICLR.

[24]  Yaochu Jin,et al.  Multi-Objective Evolutionary Federated Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Yue Ming,et al.  Deep learning for monocular depth estimation: A review , 2021, Neurocomputing.

[26]  Hongyu Li,et al.  An End-to-End Encrypted Neural Network for Gradient Updates Transmission in Federated Learning , 2019, 2019 Data Compression Conference (DCC).

[27]  Sebastian U. Stich,et al.  Ensemble Distillation for Robust Model Fusion in Federated Learning , 2020, NeurIPS.

[28]  Jie Xu,et al.  The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoost , 2019, ArXiv.

[29]  Tassilo Klein,et al.  Differentially Private Federated Learning: A Client Level Perspective , 2017, ArXiv.

[30]  Deshi Li,et al.  FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing , 2020, IEEE Open Journal of Signal Processing.

[31]  Sunav Choudhary,et al.  Federated Learning with Personalization Layers , 2019, ArXiv.

[32]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Zhong Fan,et al.  A Review of Privacy-Preserving Federated Learning for the Internet-of-Things , 2020 .

[34]  Bill Lin,et al.  Differentially-private Federated Neural Architecture Search , 2020, ArXiv.

[35]  Nguyen H. Tran,et al.  Personalized Federated Learning with Moreau Envelopes , 2020, NeurIPS.

[36]  Shengwen Yang,et al.  Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator , 2019, ArXiv.

[37]  Weishan Zhang,et al.  Heterogeneous Federated Learning , 2020, ArXiv.

[38]  Jiaya Jia,et al.  Fast and Practical Neural Architecture Search , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[39]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[40]  Amir Salman Avestimehr,et al.  FedNAS: Federated Deep Learning via Neural Architecture Search , 2020, ArXiv.

[41]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[42]  Asim Kadav,et al.  MALT: distributed data-parallelism for existing ML applications , 2015, EuroSys.

[43]  Jie Xu,et al.  Federated Learning for Healthcare Informatics , 2019, ArXiv.

[44]  Tianjian Chen,et al.  A Communication Efficient Collaborative Learning Framework for Distributed Features , 2019 .

[45]  Rich Caruana,et al.  Model compression , 2006, KDD '06.

[46]  Bo Zhao,et al.  iDLG: Improved Deep Leakage from Gradients , 2020, ArXiv.

[47]  Peter Richtárik,et al.  Federated Learning of a Mixture of Global and Local Models , 2020, ArXiv.

[48]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[49]  H. Vincent Poor,et al.  Federated Learning With Differential Privacy: Algorithms and Performance Analysis , 2019, IEEE Transactions on Information Forensics and Security.

[50]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[51]  Xiao Jin,et al.  VAFL: a Method of Vertical Asynchronous Federated Learning , 2020, ArXiv.

[52]  Kaigui Bian,et al.  Neural Architecture Search over Decentralized Data , 2020, ArXiv.

[53]  Zidong Wang,et al.  A Dynamic Neighborhood-Based Switching Particle Swarm Optimization Algorithm , 2020, IEEE Transactions on Cybernetics.

[54]  F. Rudzicz.  Human Language Technologies : The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics , 2010 .

[55]  Milind Kulkarni,et al.  Survey of Personalization Techniques for Federated Learning , 2020, 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4).

[56]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[57]  Jun Zhang,et al.  Communication-Efficient Edge AI: Algorithms and Systems , 2020, IEEE Communications Surveys & Tutorials.

[58]  Whitfield Diffie,et al.  New Directions in Cryptography , 1976, IEEE Trans. Inf. Theory.

[59]  Yuanming Shi,et al.  A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression , 2019, ArXiv.

[60]  Yaochu Jin,et al.  A Federated Data-Driven Evolutionary Algorithm , 2021, Knowl. Based Syst..

[61]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[62]  Yang Liu,et al.  Real-World Image Datasets for Federated Learning , 2019, ArXiv.

[63]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[64]  Jun Zhao,et al.  Local Differential Privacy-Based Federated Learning for Internet of Things , 2020, IEEE Internet of Things Journal.

[65]  Hubert Eichner,et al.  Federated Evaluation of On-device Personalization , 2019, ArXiv.

[66]  Chenchen Liu,et al.  Fed2: Feature-Aligned Federated Learning , 2021, KDD.

[67]  Qiang Yang,et al.  SecureBoost: A Lossless Federated Learning Framework , 2019, IEEE Intelligent Systems.

[68]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[69]  Geyong Min,et al.  Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT , 2020, IEEE Internet of Things Journal.

[70]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[71]  Yasaman Khazaeni,et al.  Federated Learning with Matched Averaging , 2020, ICLR.

[72]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[73]  Miriam Bellver,et al.  Distributed training strategies for a computer vision deep learning algorithm on a distributed GPU cluster , 2017, ICCS.

[74]  Beng Chin Ooi,et al.  Privacy preserving vertical federated learning for tree-based models , 2020, Proc. VLDB Endow..

[75]  Tzu-Ming Harry Hsu,et al.  Federated Visual Classification with Real-World Data Distribution , 2020, ECCV.

[76]  Ramesh Raskar,et al.  No Peek: A Survey of private distributed deep learning , 2018, ArXiv.

[77]  Hangyu Zhu,et al.  Toward Real-Time Federated Evolutionary Neural Architecture Search , 2021, Automated Design of Machine Learning and Search Algorithms.

[78]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[79]  Jascha Sohl-Dickstein,et al.  Measuring the Effects of Data Parallelism on Neural Network Training , 2018, J. Mach. Learn. Res..

[80]  Nadav Israel,et al.  Overcoming Forgetting in Federated Learning on Non-IID Data , 2019, ArXiv.

[81]  Yun Yang,et al.  Comparison and Modelling of Country-level Microblog User and Activity in Cyber-physical-social Systems Using Weibo and Twitter Data , 2019, ACM Trans. Intell. Syst. Technol..

[82]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[83]  Tianjian Chen,et al.  Self-supervised Cross-silo Federated Neural Architecture Search , 2021, ArXiv.

[84]  Jia Liu,et al.  Multi-objective Search of Robust Neural Architectures against Multiple Types of Adversarial Attacks , 2021, Neurocomputing.

[85]  Cynthia Dwork.  International Conference on Theory and Applications of Models of Computation , 2008 .

[86]  Sebastian Caldas,et al.  Expanding the Reach of Federated Learning by Reducing Client Resource Requirements , 2018, ArXiv.

[87]  Mehrdad Mahdavi,et al.  Adaptive Personalized Federated Learning , 2020, ArXiv.

[88]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[89]  Hao Wang,et al.  Optimizing Federated Learning on Non-IID Data with Reinforcement Learning , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[90]  Ming Liu,et al.  Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems , 2019, IEEE Robotics and Automation Letters.

[91]  Pramod K. Varshney,et al.  Decentralized Federated Learning via Mutual Knowledge Transfer , 2020, IEEE Internet of Things Journal.

[92]  Bernd G. Pfrommer,et al.  Relaxation of Crystals with the Quasi-Newton Method , 1997 .

[93]  Han Yu,et al.  Threats to Federated Learning: A Survey , 2020, ArXiv.

[94]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[95]  Yang Liu,et al.  BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning , 2020, USENIX ATC.

[96]  Craig Gentry,et al.  A fully homomorphic encryption scheme , 2009 .

[97]  Bingsheng He,et al.  Federated Learning on Non-IID Data Silos: An Experimental Study , 2021, 2022 IEEE 38th International Conference on Data Engineering (ICDE).

[98]  Richard Nock,et al.  Entity Resolution and Federated Learning get a Federated Resolution , 2018, ArXiv.

[99]  Manzil Zaheer,et al.  Adaptive Federated Optimization , 2020, ICLR.

[100]  Gideon S. Mann,et al.  Distributed Training Strategies for the Structured Perceptron , 2010, NAACL.

[101]  Cynthia Dwork,et al.  Differential Privacy: A Survey of Results , 2008, TAMC.

[102]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.

[103]  Farzin Haddadpour,et al.  On the Convergence of Local Descent Methods in Federated Learning , 2019, ArXiv.

[104]  Vitaly Shmatikov,et al.  Salvaging Federated Learning by Local Adaptation , 2020, ArXiv.

[105]  Junpu Wang,et al.  FedMD: Heterogenous Federated Learning via Model Distillation , 2019, ArXiv.

[106]  Yang Song,et al.  Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning , 2018, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[107]  Sreeram Kannan,et al.  Improving Federated Learning Personalization via Model Agnostic Meta Learning , 2019, ArXiv.

[108]  Anit Kumar Sahu,et al.  On the Convergence of Federated Optimization in Heterogeneous Networks , 2018, ArXiv.

[109]  Bingsheng He,et al.  A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection , 2019, IEEE Transactions on Knowledge and Data Engineering.

[110]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[111]  Jian Pei,et al.  Personalized Cross-Silo Federated Learning on Non-IID Data , 2020, AAAI.

[112]  J. Brian Gray,et al.  Introduction to Linear Regression Analysis , 2002, Technometrics.

[113]  Wenqi Wei,et al.  A Framework for Evaluating Client Privacy Leakages in Federated Learning , 2020, ESORICS.

[114]  Klaus-Robert Müller,et al.  Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[115]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2021, Found. Trends Mach. Learn..

[116]  Lisandro Dalcin,et al.  Parallel distributed computing using Python , 2011 .

[117]  Ming Li,et al.  Wireless Federated Learning with Local Differential Privacy , 2020, 2020 IEEE International Symposium on Information Theory (ISIT).

[118]  Kuo-Yi Lin,et al.  A Survey on federated learning* , 2020, 2020 IEEE 16th International Conference on Control & Automation (ICCA).

[119]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[120]  Fei Chen,et al.  Federated Meta-Learning with Fast Convergence and Efficient Communication , 2018 .

[121]  Tianjian Chen,et al.  A Secure Federated Transfer Learning Framework , 2020, IEEE Intelligent Systems.

[122]  Ravi Tandon,et al.  On the Capacity of Secure Distributed Matrix Multiplication , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[123]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[124]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[125]  Pascal Paillier,et al.  Public-Key Cryptosystems Based on Composite Degree Residuosity Classes , 1999, EUROCRYPT.

[126]  Ramesh Raskar,et al.  FedML: A Research Library and Benchmark for Federated Machine Learning , 2020, ArXiv.

[127]  H. Brendan McMahan,et al.  Learning Differentially Private Recurrent Language Models , 2017, ICLR.

[128]  Wenqi Wei,et al.  LDP-Fed: federated learning with local differential privacy , 2020, EdgeSys@EuroSys.

[129]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[130]  Huchuan Lu,et al.  Deep Mutual Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[131]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[132]  Wangli He,et al.  Ternary Compression for Communication-Efficient Federated Learning , 2020, IEEE transactions on neural networks and learning systems.

[133]  Jianping Gou,et al.  Knowledge Distillation: A Survey , 2020, International Journal of Computer Vision.

[134]  Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models , 2021, 2021 IEEE International Conference on Big Data (Big Data).

[135]  Cong Xu,et al.  TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning , 2017, NIPS.

[136]  Kannan Ramchandran,et al.  Robust Federated Learning in a Heterogeneous Environment , 2019, ArXiv.

[137]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[138]  M. Hadi Amini,et al.  Federated Learning for Resource-Constrained IoT Devices: Panoramas and State-of-the-art , 2020, ArXiv.

[139]  Kin K. Leung,et al.  Overcoming Noisy and Irrelevant Data in Federated Learning , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[140]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[141]  Christopher Briggs,et al.  Federated learning with hierarchical clustering of local updates to improve training on non-IID data , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[142]  Reza M. Parizi,et al.  Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications , 2020, IEEE Access.

[143]  Kai Chen,et al.  Quantifying the Performance of Federated Transfer Learning , 2019, ArXiv.

[144]  K. Ramchandran,et al.  An Efficient Framework for Clustered Federated Learning , 2020, IEEE Transactions on Information Theory.

[145]  Sanjiv Kumar,et al.  Federated Learning with Only Positive Labels , 2020, ICML.

[146]  Huaiyu Dai,et al.  From Federated Learning to Fog Learning: Towards Large-Scale Distributed Machine Learning in Heterogeneous Wireless Networks , 2020, ArXiv.

[147]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[148]  Kate Saenko,et al.  Federated Adversarial Domain Adaptation , 2020, ICLR.

[149]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning: Revisited and Enhanced , 2017, ATIS.

[150]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[151]  Amir Houmansadr,et al.  Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer , 2019, ArXiv.

[152]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2018, IEEE Transactions on Information Forensics and Security.

[153]  Hangyu Zhu,et al.  Real-Time Federated Evolutionary Neural Architecture Search , 2020, IEEE Transactions on Evolutionary Computation.

[154]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[155]  Frederik Armknecht,et al.  A Guide to Fully Homomorphic Encryption , 2015, IACR Cryptol. ePrint Arch..

[156]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[157]  Haomiao Yang,et al.  Towards Efficient and Privacy-Preserving Federated Deep Learning , 2019, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[158]  Ruslan Salakhutdinov,et al.  Think Locally, Act Globally: Federated Learning with Local and Global Representations , 2020, ArXiv.

[159]  Wright-Patterson Afb,et al.  Feature Selection Using a Multilayer Perceptron , 1990 .

[160]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[161]  Hangyu Zhu,et al.  From federated learning to federated neural architecture search: a survey , 2020, Complex & Intelligent Systems.

[162]  Zihao Wang,et al.  PV-NAS: Practical Neural Architecture Search for Video Recognition , 2020, ArXiv.

[163]  Kuangrong Hao,et al.  Efficient Evolutionary Search of Attention Convolutional Networks via Sampled Training and Node Inheritance , 2020, IEEE Transactions on Evolutionary Computation.

[164]  Seong-Lyun Kim,et al.  XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learning , 2020, ArXiv.

[165]  Sergey Ioffe,et al.  Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models , 2017, NIPS.

[166]  Moming Duan,et al.  Astraea: Self-Balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications , 2019, 2019 IEEE 37th International Conference on Computer Design (ICCD).

[167]  Joachim M. Buhmann,et al.  Variational Federated Multi-Task Learning , 2019, ArXiv.

[168]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[169]  M. W Gardner,et al.  Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences , 1998 .

[170]  Giuseppe Ateniese,et al.  Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning , 2017, CCS.

[171]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[172]  Mario A. Storti,et al.  MPI for Python: Performance improvements and MPI-2 extensions , 2008, J. Parallel Distributed Comput..

[173]  Zezhong Zhang,et al.  An Introduction to Communication Efficient Edge Machine Learning , 2019, ArXiv.

[174]  Y. Mansour,et al.  Three Approaches for Personalization with Applications to Federated Learning , 2020, ArXiv.

[175]  Hangyu Zhu,et al.  Distributed Additive Encryption and Quantization for Privacy Preserving Federated Deep Learning , 2020, Neurocomputing.

[176]  Aryan Mokhtari,et al.  Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach , 2020, NeurIPS.

[177]  Emiliano De Cristofaro,et al.  Toward Robustness and Privacy in Federated Learning: Experimenting with Local and Central Differential Privacy , 2020, ArXiv.

[178]  Gregory Cohen,et al.  EMNIST: an extension of MNIST to handwritten letters , 2017, CVPR 2017.

[179]  Kannan Ramchandran,et al.  Fast Distributed Training of Deep Neural Networks: Dynamic Communication Thresholding for Model and Data Parallelism , 2020, ArXiv.

[180]  Fuad E. Alsaadi,et al.  Deep-reinforcement-learning-based images segmentation for quantitative analysis of gold immunochromatographic strip , 2020, Neurocomputing.

[181]  Dong Yu,et al.  1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs , 2014, INTERSPEECH.