The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning

The Internet of Things (IoT) is on the verge of a major paradigm shift. In the IoT system of the future, IoFT, the “cloud” will be substituted by the “crowd” where model training is brought to the edge, allowing IoT devices to collaboratively extract knowledge and build smart analytics/models while keeping their personal data stored locally. This paradigm shift was set into motion by the tremendous increase in computational power on IoT devices and the recent advances in decentralized and privacypreserving model training, coined as federated learning (FL). This article provides a vision for IoFT and a systematic overview of current efforts towards realizing this vision. Specifically, we first introduce the defining characteristics of IoFT and discuss FL data-driven approaches, opportunities, and challenges that allow decentralized inference within three dimensions: (i) a global model that maximizes utility across all IoT devices, (ii) a personalized model that borrows strengths across all devices yet retains its own model, (iii) a meta-learning model that quickly adapts to new devices or learning tasks. We end by describing the vision and challenges of IoFT in reshaping different industries through the lens of domain experts. Those industries include manufacturing, transportation, energy, healthcare, quality & reliability, business, and computing. INDEX TERMS Internet of Things, Federated learning, Global Model, Personalized Model, MetaLearning, Future Applications.

[1]  H. H. Rachford,et al.  The Numerical Solution of Parabolic and Elliptic Differential Equations , 1955 .

[2]  Y. Gordon.  Some inequalities for Gaussian processes and applications , 1985 .

[3]  Sanjay Srivastava,et al.  On Repeated Moral Hazard with Discounting , 1987 .

[4]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[5]  J. Bert Keats,et al.  Statistical Methods for Reliability Data , 1999 .

[6]  P. Bickel,et al.  Non- and semiparametric statistics: compared and contrasted , 2000 .

[7]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[8]  Petros Drineas,et al.  On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning , 2005, J. Mach. Learn. Res..

[9]  Yuliy Sannikov.  A Continuous-Time Version of the Principal-Agent , 2005 .

[10]  Ann Q. Gates,et al.  TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING , 2005 .

[11]  J. Zimmerman,et al.  Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients* , 2006, Critical care medicine.

[12]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[13]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[14]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[15]  Francis R. Bach,et al.  Consistency of the group Lasso and multiple kernel learning , 2007, J. Mach. Learn. Res..

[16]  Douglas MacFadden,et al.  Application of Information Technology The Shared Health Research Information Network ( SHRINE ) : A Prototype Federated Query Tool for Clinical Data Repositories , 2014 .

[17]  Larry A. Wasserman,et al.  The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs , 2009, J. Mach. Learn. Res..

[18]  Gideon S. Mann,et al.  Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models , 2009, NIPS.

[19]  Petros Drineas,et al.  CUR matrix decompositions for improved data analysis , 2009, Proceedings of the National Academy of Sciences.

[20]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[21]  Wei Chen,et al.  Optimizing Latin hypercube design for sequential sampling of computer experiments , 2009 .

[22]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[23]  Alexander J. Smola,et al.  Parallelized Stochastic Gradient Descent , 2010, NIPS.

[24]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[25]  John L. Nazareth,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[26]  Dit-Yan Yeung,et al.  A Convex Formulation for Learning Task Relationships in Multi-Task Learning , 2010, UAI.

[27]  V. Koltchinskii,et al.  SPARSITY IN MULTIPLE KERNEL LEARNING , 2010, 1211.2998.

[28]  Michael I. Jordan,et al.  Managing data transfers in computer clusters with orchestra , 2011, SIGCOMM.

[29]  S. Muthukrishnan,et al.  Faster least squares approximation , 2007, Numerische Mathematik.

[30]  Yee Whye Teh,et al.  Bayesian Learning via Stochastic Gradient Langevin Dynamics , 2011, ICML.

[31]  Kristen Grauman,et al.  Learning with Whom to Share in Multi-task Feature Learning , 2011, ICML.

[32]  Martin J. Wainwright,et al.  Early stopping for non-parametric regression: An optimal data-dependent stopping rule , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[33]  Shuicheng Yan,et al.  Visual Classification With Multitask Joint Sparse Representation , 2012, IEEE Transactions on Image Processing.

[34]  J. New,et al.  Evaluation of weather datasets for building energy simulation , 2012 .

[35]  R. Wiser,et al.  Renewable Electricity Futures Study. Executive Summary , 2012 .

[36]  Sébastien Bubeck,et al.  Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , 2012, Found. Trends Mach. Learn..

[37]  Martin J. Wainwright,et al.  Minimax-Optimal Rates For Sparse Additive Models Over Kernel Classes Via Convex Programming , 2010, J. Mach. Learn. Res..

[38]  Hal Daumé,et al.  Learning Task Grouping and Overlap in Multi-task Learning , 2012, ICML.

[39]  Motoaki Kawanabe,et al.  Machine Learning in Non-Stationary Environments - Introduction to Covariate Shift Adaptation , 2012, Adaptive computation and machine learning.

[40]  Hari Balakrishnan,et al.  TCP ex machina: computer-generated congestion control , 2013, SIGCOMM.

[41]  Nikolaos V. Sahinidis,et al.  Derivative-free optimization: a review of algorithms and comparison of software implementations , 2013, J. Glob. Optim..

[42]  Z. Ye,et al.  How do heterogeneities in operating environments affect field failure predictions and test planning , 2013, 1401.2282.

[43]  Jian Liu,et al.  Diagnosing Multistage Manufacturing Processes With Engineering-Driven Factor Analysis Considering Sampling Uncertainty , 2013 .

[44]  Luciano Messori.  The Theory of Incentives I: The Principal-Agent Model , 2013 .

[45]  William H. Woodall,et al.  An overview of George Box's contributions to process monitoring and feedback adjustment , 2014 .

[46]  Damian Flynn,et al.  Variable Generation, Reserves, Flexibility and Policy Interactions , 2014, 2014 47th Hawaii International Conference on System Sciences.

[47]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[48]  Jakob Stoustrup,et al.  Integration of Flexible Consumers in the Ancillary Service Markets , 2014 .

[49]  M. Porter,et al.  How Smart, Connected Products Are Transforming Competition , 2014 .

[50]  Ohad Shamir,et al.  Communication-Efficient Distributed Optimization using an Approximate Newton-type Method , 2013, ICML.

[51]  Eunshin Byon,et al.  Condition Monitoring of Wind Power System With Nonparametric Regression Analysis , 2014, IEEE Transactions on Energy Conversion.

[52]  Xi Zhang,et al.  Adaptive Sensor Allocation Strategy for Process Monitoring and Diagnosis in a Bayesian Network , 2014, IEEE Transactions on Automation Science and Engineering.

[53]  Trevor Hastie,et al.  Statistical Learning with Sparsity: The Lasso and Generalizations , 2015 .

[54]  Athanasios V. Vasilakos,et al.  Data Mining for the Internet of Things: Literature Review and Challenges , 2015, Int. J. Distributed Sens. Networks.

[55]  Shreyes N. Melkote,et al.  Analysis of Computer Experiments With Functional Response , 2012, Technometrics.

[56]  Carlos Eduardo Scheidegger,et al.  Certifying and Removing Disparate Impact , 2014, KDD.

[57]  Garvesh Raskutti,et al.  Learning Large-Scale Poisson DAG Models based on OverDispersion Scoring , 2015, NIPS.

[58]  Mo Dong,et al.  PCC: Re-architecting Congestion Control for Consistent High Performance , 2014, NSDI.

[59]  Anne-Marie Kermarrec,et al.  Want to scale in centralized systems? Think P2P , 2015, Journal of Internet Services and Applications.

[60]  Paramvir Bahl,et al.  Low Latency Geo-distributed Data Analytics , 2015, SIGCOMM.

[61]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[62]  Mumbai,et al.  Internet of Things (IoT): A Literature Review , 2015 .

[63]  Griffin M. Weber.  Federated queries of clinical data repositories: Scaling to a national network , 2015, J. Biomed. Informatics.

[64]  Lihui Wang,et al.  Cloud Manufacturing: Current Trends and Future Implementations , 2015 .

[65]  Lida Xu,et al.  The internet of things: a survey , 2014, Information Systems Frontiers.

[66]  Karl Henrik Johansson,et al.  A Distributed Framework for Coordinated Heavy-Duty Vehicle Platooning , 2015, IEEE Transactions on Intelligent Transportation Systems.

[67]  Michael Lees,et al.  A Partition-Based Match Making Algorithm for Dynamic Ridesharing , 2015, IEEE Transactions on Intelligent Transportation Systems.

[68]  Zhen Shao,et al.  Energy Internet: The business perspective , 2016 .

[69]  Michael W. Mahoney,et al.  A Statistical Perspective on Randomized Sketching for Ordinary Least-Squares , 2014, J. Mach. Learn. Res..

[70]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[71]  Jorge Nocedal,et al.  A Stochastic Quasi-Newton Method for Large-Scale Optimization , 2014, SIAM J. Optim..

[72]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[73]  Dilin Wang,et al.  Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm , 2016, NIPS.

[74]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[75]  Jörg Krüger,et al.  Feasibility of connecting machinery and robots to industrial control services in the cloud , 2016, 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA).

[76]  Eunshin Byon,et al.  Adaptive Learning in Time-Variant Processes With Application to Wind Power Systems , 2016, IEEE Transactions on Automation Science and Engineering.

[77]  Fiona Charnley,et al.  Distributed manufacturing: scope, challenges and opportunities , 2016 .

[78]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[79]  Chongqing Kang,et al.  Evaluating the Contribution of Energy Storages to Support Large-Scale Renewable Generation in Joint Energy and Ancillary Service Markets , 2016, IEEE Transactions on Sustainable Energy.

[80]  Fernando José Von Zuben,et al.  Multi-task Sparse Structure Learning with Gaussian Copula Models , 2016, J. Mach. Learn. Res..

[81]  Jingjing Li,et al.  Change‐Point Detection on Solar Panel Performance Using Thresholded LASSO , 2016, Qual. Reliab. Eng. Int..

[82]  Hongzi Mao,et al.  Neural Adaptive Video Streaming with Pensieve , 2017, SIGCOMM.

[83]  Eunshin Byon,et al.  When wind travels through turbines: A new statistical approach for characterizing heterogeneous wake effects in multi-turbine wind farms , 2017 .

[84]  Günther R. Raidl,et al.  Full‐load route planning for balancing bike sharing systems by logic‐based benders decomposition , 2017, Networks.

[85]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[86]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[87]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[88]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[89]  Zhi-Sheng Ye,et al.  Random Effects Models for Aggregate Lifetime Data , 2017, IEEE Transactions on Reliability.

[90]  Wei Zhang,et al.  Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent , 2017, NIPS.

[91]  Amos J. Storkey,et al.  Towards a Neural Statistician , 2016, ICLR.

[92]  Krishna P. Gummadi,et al.  Fairness Constraints: Mechanisms for Fair Classification , 2015, AISTATS.

[93]  Xi Chen,et al.  Evolution Strategies as a Scalable Alternative to Reinforcement Learning , 2017, ArXiv.

[94]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[95]  Úlfar Erlingsson,et al.  Prochlo: Strong Privacy for Analytics in the Crowd , 2017, SOSP.

[96]  Elie Bou-Zeid,et al.  Surface heat assessment for developed environments: Probabilistic urban temperature modeling , 2017, Comput. Environ. Urban Syst..

[97]  Zhi-Sheng Ye,et al.  Estimation of Field Reliability Based on Aggregate Lifetime Data , 2017, Technometrics.

[98]  Janardhan Kulkarni,et al.  Collecting Telemetry Data Privately , 2017, NIPS.

[99]  M. Plumlee.  Bayesian Calibration of Inexact Computer Models , 2017 .

[100]  Mark W. Schmidt,et al.  Minimizing finite sums with the stochastic average gradient , 2013, Mathematical Programming.

[101]  Dafna Shahaf,et al.  Learning to Route , 2017, HotNets.

[102]  Eunshin Byon,et al.  A sparse partitioned-regression model for nonlinear system–environment interactions , 2017 .

[103]  Weichung Wang,et al.  Sequential Designs Based on Bayesian Uncertainty Quantification in Sparse Representation Surrogate Modeling , 2017, Technometrics.

[104]  Saad Mubeen,et al.  Delay Mitigation in Offloaded Cloud Controllers in Industrial IoT , 2017, IEEE Access.

[105]  Jinfeng Yi,et al.  ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models , 2017, AISec@CCS.

[106]  R. Jayakrishnan,et al.  A Decomposition Algorithm to Solve the Multi-Hop Peer-to-Peer Ride-Matching Problem , 2017, ArXiv.

[107]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[108]  D. Roller,et al.  Additive Manufacturing , Cloud-Based 3 D Printing and Associated Services — Overview , 2017 .

[109]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[110]  Dawn Xiaodong Song,et al.  Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning , 2017, ArXiv.

[111]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[112]  Hang Li,et al.  Meta-SGD: Learning to Learn Quickly for Few Shot Learning , 2017, ArXiv.

[113]  Bernhard Schölkopf,et al.  Discriminative k-shot learning using probabilistic models , 2017, ArXiv.

[114]  J. Popp,et al.  The Role and Impact of Industry 4.0 and the Internet of Things on the Business Strategy of the Value Chain—The Case of Hungary , 2018, Sustainability.

[115]  Fengqi You,et al.  Sustainable Manufacturing With Cyber-Physical Discrete Manufacturing Networks: Overview and Modeling Framework , 2019, Journal of Manufacturing Science and Engineering.

[116]  Bin Gu,et al.  Training Neural Networks Using Features Replay , 2018, NeurIPS.

[117]  Edward A. Lee,et al.  AWStream: adaptive wide-area streaming analytics , 2018, SIGCOMM.

[118]  Patrick L. Combettes,et al.  Monotone operator theory in convex optimization , 2018, Math. Program..

[119]  Ion Stoica,et al.  Chameleon: scalable adaptation of video analytics , 2018, SIGCOMM.

[120]  C. Okwudire,et al.  A limited-preview filtered B-spline approach to tracking control – With application to vibration-induced error compensation of a 3D printer , 2017, Mechatronics.

[121]  Sergey Levine,et al.  Probabilistic Model-Agnostic Meta-Learning , 2018, NeurIPS.

[122]  S. Kardia,et al.  Public Trust in Health Information Sharing: A Measure of System Trust , 2018, Health services research.

[123]  Zhenguo Li,et al.  Federated Meta-Learning with Fast Convergence and Efficient Communication , 2018, 1802.07876.

[124]  François Fleuret,et al.  Not All Samples Are Created Equal: Deep Learning with Importance Sampling , 2018, ICML.

[125]  Ching Chuen Chan,et al.  Enabling Industrial Internet of Things (IIoT) towards an emerging smart energy system , 2018 .

[126]  Thomas L. Griffiths,et al.  Recasting Gradient-Based Meta-Learning as Hierarchical Bayes , 2018, ICLR.

[127]  Pengfei Wei,et al.  Reliability and reliability-based importance analysis of structural systems using multiple response Gaussian process model , 2018, Reliab. Eng. Syst. Saf..

[128]  Paramvir Bahl,et al.  Focus: Querying Large Video Datasets with Low Latency and Low Cost , 2018, OSDI.

[129]  Max Tegmark,et al.  Meta-learning autoencoders for few-shot prediction , 2018, ArXiv.

[130]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[131]  Placid Mathew Ferreira,et al.  A new paradigm for organizing networks of computer numerical control manufacturing resources in cloud manufacturing , 2018 .

[132]  J. Schulman,et al.  Reptile: a Scalable Metalearning Algorithm , 2018 .

[133]  Le Minh Kieu,et al.  Deep learning methods in transportation domain: a review , 2018, IET Intelligent Transport Systems.

[134]  Fred Feng,et al.  Estimation of Lead Vehicle Kinematics Using Camera-Based Data for Driver Distraction Detection , 2018 .

[135]  Gaurav Kapoor,et al.  Protection Against Reconstruction and Its Applications in Private Federated Learning , 2018, ArXiv.

[136]  Alexandre Lacoste,et al.  Uncertainty in Multitask Transfer Learning , 2018, ArXiv.

[137]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[138]  Meng Zhao,et al.  Ridesharing Problem with Flexible Pickup and Delivery Locations for App-Based Transportation Service: Mathematical Modeling and Decomposition Methods , 2018, Journal of Advanced Transportation.

[139]  Gengyin Li,et al.  Optimal residential community demand response scheduling in smart grid , 2018 .

[140]  Yoshua Bengio,et al.  Bayesian Model-Agnostic Meta-Learning , 2018, NeurIPS.

[141]  C. Okwudire,et al.  Low-Level Control of 3D Printers from the Cloud:  A Step Toward 3D Printer Control as a Service , 2018 .

[142]  Richard Nock,et al.  Entity Resolution and Federated Learning get a Federated Resolution , 2018, ArXiv.

[143]  Wei Shi,et al.  Federated learning of predictive models from federated Electronic Health Records , 2018, Int. J. Medical Informatics.

[144]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[145]  Sanjiv Kumar,et al.  Adaptive Methods for Nonconvex Optimization , 2018, NeurIPS.

[146]  Rachid Guerraoui,et al.  Personalized and Private Peer-to-Peer Machine Learning , 2017, AISTATS.

[147]  Shiyu Zhou,et al.  Nonparametric Modeling and Prognosis of Condition Monitoring Signals Using Multivariate Gaussian Convolution Processes , 2018, Technometrics.

[148]  Xudong Jiang,et al.  Deep Coupled ResNet for Low-Resolution Face Recognition , 2018, IEEE Signal Processing Letters.

[149]  Yee Whye Teh,et al.  Conditional Neural Processes , 2018, ICML.

[150]  Sepp Hochreiter,et al.  First Order Generative Adversarial Networks , 2018, ICML.

[151]  Bruce R. Rosen,et al.  Distributed deep learning networks among institutions for medical imaging , 2018, J. Am. Medical Informatics Assoc..

[152]  Hubert Eichner,et al.  Federated Learning for Mobile Keyboard Prediction , 2018, ArXiv.

[153]  Andrew Gordon Wilson,et al.  Averaging Weights Leads to Wider Optima and Better Generalization , 2018, UAI.

[154]  Hubert Eichner,et al.  APPLIED FEDERATED LEARNING: IMPROVING GOOGLE KEYBOARD QUERY SUGGESTIONS , 2018, ArXiv.

[155]  Shiyu Zhou,et al.  Nonparametric-Condition-Based Remaining Useful Life Prediction Incorporating External Factors , 2018, IEEE Transactions on Reliability.

[156]  Joaquin Vanschoren,et al.  Meta-Learning: A Survey , 2018, Automated Machine Learning.

[157]  Tim Kraska,et al.  The Case for Learned Index Structures , 2018 .

[158]  Mariagrazia Dotoli,et al.  Decentralized control for residential energy management of a smart users ʼ microgrid with renewable energy exchange , 2019, IEEE/CAA Journal of Automatica Sinica.

[159]  Razvan Pascanu,et al.  Meta-Learning with Latent Embedding Optimization , 2018, ICLR.

[160]  Raed Kontar,et al.  Variational Inference of Joint Models using Multivariate Gaussian Convolution Processes , 2019, ArXiv.

[161]  Mehryar Mohri,et al.  Agnostic Federated Learning , 2019, ICML.

[162]  Swaroop Ramaswamy,et al.  Federated Learning for Emoji Prediction in a Mobile Keyboard , 2019, ArXiv.

[163]  Moming Duan,et al.  Astraea: Self-Balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications , 2019, 2019 IEEE 37th International Conference on Computer Design (ICCD).

[164]  Alex Beatson,et al.  Amortized Bayesian Meta-Learning , 2018, ICLR.

[165]  Rasool Mohebifard,et al.  Optimal network-level traffic signal control: A benders decomposition-based solution algorithm , 2019, Transportation Research Part B: Methodological.

[166]  William R. Myers,et al.  Space-Filling Designs for Robustness Experiments , 2017, Technometrics.

[167]  Hubert Eichner,et al.  Federated Evaluation of On-device Personalization , 2019, ArXiv.

[168]  Sunav Choudhary,et al.  Federated Learning with Personalization Layers , 2019, ArXiv.

[169]  Andrew Gordon Wilson,et al.  A Simple Baseline for Bayesian Uncertainty in Deep Learning , 2019, NeurIPS.

[170]  M. Plumlee.  Computer model calibration with confidence and consistency , 2019, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[171]  Nan Xu,et al.  CoLight: Learning Network-level Cooperation for Traffic Signal Control , 2019, CIKM.

[172]  Bing Ren,et al.  Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator , 2019, ArXiv.

[173]  Andrew A. Chien,et al.  Networked Cameras Are the New Big Data Clusters , 2019, HotEdgeVideo@MOBICOM.

[174]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.

[175]  Kristen S. Cetin,et al.  Projecting the Most Likely Annual Urban Heat Extremes in the Central United States , 2019, Atmosphere.

[176]  Ji Liu,et al.  DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression , 2019, ICML.

[177]  Yasaman Khazaeni,et al.  Bayesian Nonparametric Federated Learning of Neural Networks , 2019, ICML.

[178]  Surya R. Kalidindi,et al.  Bayesian Sequential Design of Experiments for Extraction of Single-Crystal Material Properties from Spherical Indentation Measurements on Polycrystalline Samples , 2019, JOM.

[179]  Jun Wang,et al.  Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning , 2019, WWW.

[180]  Anit Kumar Sahu,et al.  FedDANE: A Federated Newton-Type Method , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[181]  Xu Chen,et al.  In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning , 2018, IEEE Network.

[182]  Ilana Segall,et al.  Federated Learning for Ranking Browser History Suggestions , 2019, ArXiv.

[183]  Klamer Schutte,et al.  The Functional Neural Process , 2019, NeurIPS.

[184]  Alexandre Lacoste,et al.  Adaptive Deep Kernel Learning , 2019, ArXiv.

[185]  Sebastian Nowozin,et al.  Meta-Learning Probabilistic Inference for Prediction , 2018, ICLR.

[186]  The Renyi Gaussian Process: Towards Improved Generalization , 2019, 1910.06990.

[187]  Qiang Yang,et al.  Real-World Image Datasets for Federated Learning , 2019, ArXiv.

[188]  Cyril Allauzen,et al.  Federated Learning of N-Gram Language Models , 2019, CoNLL.

[189]  Brighten Godfrey,et al.  A Deep Reinforcement Learning Perspective on Internet Congestion Control , 2019, ICML.

[190]  Daguang Xu,et al.  Privacy-preserving Federated Brain Tumour Segmentation , 2019, MLMI@MICCAI.

[191]  Shengli Xie,et al.  Incentive Mechanism for Reliable Federated Learning: A Joint Optimization Approach to Combining Reputation and Contract Theory , 2019, IEEE Internet of Things Journal.

[192]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[193]  Hedvig Kjellström,et al.  Advances in Variational Inference , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[194]  Yee Whye Teh,et al.  Attentive Neural Processes , 2019, ICLR.

[195]  Yu-Chiang Frank Wang,et al.  A Closer Look at Few-shot Classification , 2019, ICLR.

[196]  Xuanjing Huang,et al.  How to Fine-Tune BERT for Text Classification? , 2019, CCL.

[197]  Aryan Mokhtari,et al.  Robust and Communication-Efficient Collaborative Learning , 2019, NeurIPS.

[198]  Yafeng Yin,et al.  Vehicle-to-vehicle wireless power transfer: Paving the way toward an electrified transportation system , 2019, Transportation Research Part C: Emerging Technologies.

[199]  G. Collins,et al.  PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies , 2019, Annals of Internal Medicine.

[200]  Zifa Liu,et al.  Optimal operation of independent regional power grid with multiple wind-solar-hydro-battery power , 2019, Applied Energy.

[201]  Ramesh Raskar,et al.  Detailed comparison of communication efficiency of split learning and federated learning , 2019, ArXiv.

[202]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[203]  Elliot J. Crowley,et al.  Deep Kernel Transfer in Gaussian Processes for Few-shot Learning , 2019, ArXiv.

[204]  Sreeram Kannan,et al.  Improving Federated Learning Personalization via Model Agnostic Meta Learning , 2019, ArXiv.

[205]  José Miguel Hernández-Lobato,et al.  Variational Implicit Processes , 2018, ICML.

[206]  Tom Ouyang,et al.  Federated Learning Of Out-Of-Vocabulary Words , 2019, ArXiv.

[207]  Stefan Winkler,et al.  The Unusual Effectiveness of Averaging in GAN Training , 2018, ICLR.

[208]  Abdul Hafeez,et al.  COVID-ResNet: A Deep Learning Framework for Screening of COVID19 from Radiographs , 2020, ArXiv.

[209]  Vitaly Shmatikov,et al.  How To Backdoor Federated Learning , 2018, AISTATS.

[210]  Chinedum E. Okwudire,et al.  A three-tier redundant architecture for safe and reliable cloud-based CNC over public internet networks , 2020, Robotics Comput. Integr. Manuf..

[211]  Elliot J. Crowley,et al.  Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels , 2020, NeurIPS.

[212]  Ali Hajbabaie,et al.  A real-time network-level traffic signal control methodology with partial connected vehicle information , 2020 .

[213]  Meisam Razaviyayn,et al.  Rényi Fair Inference , 2019, ICLR.

[214]  Vitaly Shmatikov,et al.  Salvaging Federated Learning by Local Adaptation , 2020, ArXiv.

[215]  Kristen S. Cetin,et al.  On the long-term density prediction of peak electricity load with demand side management in buildings , 2020, Energy and Buildings.

[216]  Ziyi Kou,et al.  FairFL: A Fair Federated Learning Approach to Reducing Demographic Bias in Privacy-Sensitive Classification Models , 2020, 2020 IEEE International Conference on Big Data (Big Data).

[217]  Solmaz Niknam,et al.  Federated Learning for Wireless Communications: Motivation, Opportunities, and Challenges , 2019, IEEE Communications Magazine.

[218]  Y. Mansour,et al.  Three Approaches for Personalization with Applications to Federated Learning , 2020, ArXiv.

[219]  Tengyu Ma,et al.  Federated Accelerated Stochastic Gradient Descent , 2020, NeurIPS.

[220]  Peihua Qiu,et al.  Big Data? Statistical Process Control Can Help! , 2020 .

[221]  Mehrdad Mahdavi,et al.  Adaptive Personalized Federated Learning , 2020, ArXiv.

[222]  Milind Kulkarni,et al.  Survey of Personalization Techniques for Federated Learning , 2020, 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4).

[223]  Yayi Zou,et al.  Gradient-EM Bayesian Meta-learning , 2020, NeurIPS.

[224]  Weiwen Peng,et al.  Bayesian Deep-Learning-Based Health Prognostics Toward Prognostics Uncertainty , 2020, IEEE Transactions on Industrial Electronics.

[225]  Wotao Yin,et al.  FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data , 2020, ArXiv.

[226]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[227]  Eunshin Byon,et al.  HEAT - Human Embodied Autonomous Thermostat , 2020 .

[228]  M. Chowdhury,et al.  Oort: Informed Participant Selection for Scalable Federated Learning , 2020, ArXiv.

[229]  Robert B. Gramacy,et al.  Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences , 2020 .

[230]  Martin Jaggi,et al.  Decentralized Deep Learning with Arbitrary Communication Compression , 2019, ICLR.

[231]  Wei Yang Bryan Lim,et al.  Federated Learning in Mobile Edge Networks: A Comprehensive Survey , 2019, IEEE Communications Surveys & Tutorials.

[232]  Li Huang,et al.  LoAdaBoost: Loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data , 2018, PloS one.

[233]  Peter Richtárik,et al.  Federated Learning of a Mixture of Global and Local Models , 2020, ArXiv.

[234]  Badrish Chandramouli,et al.  ALEX: An Updatable Adaptive Learned Index , 2019, SIGMOD Conference.

[235]  Amos Beimel,et al.  The power of synergy in differential privacy: Combining a small curator with local randomizers , 2019, ITC.

[236]  Nguyen H. Tran,et al.  Personalized Federated Learning with Moreau Envelopes , 2020, NeurIPS.

[237]  E. Byon,et al.  Probabilistic Characterization of Wind Diurnal Variability for Wind Resource Assessment , 2020, IEEE Transactions on Sustainable Energy.

[238]  Martin J. Wainwright,et al.  FedSplit: An algorithmic framework for fast federated optimization , 2020, NeurIPS.

[239]  Harsha V. Madhyastha,et al.  Remotely Controlled Manufacturing: A New Frontier for Systems Research , 2020, HotMobile.

[240]  Tianrui Li,et al.  Fairness and Accuracy in Federated Learning , 2020, ArXiv.

[241]  Tian Li,et al.  Fair Resource Allocation in Federated Learning , 2019, ICLR.

[242]  Yasaman Khazaeni,et al.  Federated Learning with Matched Averaging , 2020, ICLR.

[243]  Dusit Niyato,et al.  Resource Allocation in Mobility-Aware Federated Learning Networks: A Deep Reinforcement Learning Approach , 2019, 2020 IEEE 6th World Forum on Internet of Things (WF-IoT).

[244]  Hao Chen,et al.  Stochastic Gradient Descent in Correlated Settings: A Study on Gaussian Processes , 2020, NeurIPS.

[245]  Eujin Pei,et al.  3D Printing in COVID-19: Productivity Estimation of the Most Promising Open Source Solutions in Emergency Situations , 2020, Applied Sciences.

[246]  Brett K. Beaulieu-Jones,et al.  International electronic health record-derived COVID-19 clinical course profiles: the 4CE consortium , 2020, npj Digital Medicine.

[247]  Sashank J. Reddi,et al.  SCAFFOLD: Stochastic Controlled Averaging for Federated Learning , 2019, ICML.

[248]  H. Vincent Poor,et al.  Federated Learning With Differential Privacy: Algorithms and Performance Analysis , 2019, IEEE Transactions on Information Forensics and Security.

[249]  Raed Kontar,et al.  On Negative Transfer and Structure of Latent Functions in Multi-output Gaussian Processes , 2020, ArXiv.

[250]  Yaoliang Yu,et al.  FedMGDA+: Federated Learning meets Multi-objective Optimization , 2020, ArXiv.

[251]  Tianyi Wu,et al.  Adam with Bandit Sampling for Deep Learning , 2020, NeurIPS.

[252]  Li Chen,et al.  Accelerating Federated Learning via Momentum Gradient Descent , 2019, IEEE Transactions on Parallel and Distributed Systems.

[253]  Ruslan Salakhutdinov,et al.  Think Locally, Act Globally: Federated Learning with Local and Global Representations , 2020, ArXiv.

[254]  Anit Kumar Sahu,et al.  Federated Optimization in Heterogeneous Networks , 2018, MLSys.

[255]  Gustavo Carneiro,et al.  PAC-Bayesian Meta-learning with Implicit Prior , 2020, ArXiv.

[256]  Peter Richtárik,et al.  Optimal Client Sampling for Federated Learning , 2020, ArXiv.

[257]  Walid Saad,et al.  Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications , 2018, IEEE Transactions on Communications.

[258]  Francesco D’Ettorre,et al.  On the assessment and control optimisation of demand response programs in residential buildings , 2020, Renewable and Sustainable Energy Reviews.

[259]  Zhi-Sheng Ye,et al.  Parametric analysis of time-censored aggregate lifetime data , 2019, IISE Trans..

[260]  Reza M. Parizi,et al.  Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications , 2020, IEEE Access.

[261]  Jianyu Wang,et al.  Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies , 2020, ArXiv.

[262]  Behnam Zakeri,et al.  Internet of Things (IoT) and the Energy Sector , 2020, Energies.

[263]  Philip Levis,et al.  Learning in situ: a randomized experiment in video streaming , 2019, NSDI.

[264]  Spyridon Bakas,et al.  Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data , 2020, Scientific Reports.

[265]  Han Yu,et al.  Threats to Federated Learning: A Survey , 2020, ArXiv.

[266]  Yang Zhao,et al.  Bayesian Meta Sampling for Fast Uncertainty Adaptation , 2020, ICLR.

[267]  Aryan Mokhtari,et al.  Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach , 2020, NeurIPS.

[268]  R. Saigal,et al.  Dynamic Multiagent Incentive Contracts: Existence, Uniqueness, and Implementation , 2020, Prime Archives in Applied Mathematics.

[269]  Junshan Zhang,et al.  A Collaborative Learning Framework via Federated Meta-Learning , 2020, 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS).

[270]  Neda Masoud,et al.  Trip-based graph partitioning in dynamic ridesharing , 2020 .

[271]  Timothy M. Hospedales,et al.  Meta-Learning in Neural Networks: A Survey , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[272]  Fugee Tsung,et al.  Distribution inference from early-stage stationary data streams by transfer learning , 2021, IISE Transactions.

[273]  Jinny G. Park,et al.  External Validations of Cardiovascular Clinical Prediction Models: A Large-Scale Review of the Literature , 2021, medRxiv.

[274]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2019, Found. Trends Mach. Learn..

[275]  Wojciech Samek,et al.  Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[276]  Virginia Smith,et al.  Ditto: Fair and Robust Federated Learning Through Personalization , 2020, ICML.

[277]  Tinne Tuytelaars,et al.  A Continual Learning Survey: Defying Forgetting in Classification Tasks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[278]  C. Okwudire,et al.  Distributed manufacturing for and by the masses , 2021, Science.

[279]  Manzil Zaheer,et al.  Adaptive Federated Optimization , 2020, ICLR.

[280]  R. Saigal,et al.  Integrative Density Forecast and Uncertainty Quantification of Wind Power Generation , 2020, IEEE Transactions on Sustainable Energy.

[281]  Mohsen Guizani,et al.  A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond , 2021, IEEE Internet of Things Journal.

[282]  Venkatesh Saligrama,et al.  Federated Learning Based on Dynamic Regularization , 2021, ICLR.

[283]  Mariagrazia Dotoli,et al.  Robust Optimal Energy Management of a Residential Microgrid Under Uncertainties on Demand and Renewable Power Generation , 2021, IEEE Transactions on Automation Science and Engineering.

[284]  Hong-You Chen,et al.  FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning , 2020, ICLR.

[285]  Improving Fairness via Federated Learning , 2021, ArXiv.

[286]  Ping Wang,et al.  Fast-Convergent Federated Learning with Adaptive Weighting , 2020, ICC 2021 - IEEE International Conference on Communications.

[287]  E. Topol,et al.  AI-facilitated health care requires education of clinicians , 2021, The Lancet.

[288]  Shiyu Zhou,et al.  Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[289]  Jie Xu,et al.  Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective , 2020, IEEE Transactions on Wireless Communications.

[290]  Daguang Xu,et al.  Federated learning improves site performance in multicenter deep learning without data sharing , 2021, J. Am. Medical Informatics Assoc..

[291]  Hanghang Tong,et al.  Fairness-aware Agnostic Federated Learning , 2020, SDM.

[292]  Maher Nouiehed,et al.  GIFAIR-FL: An Approach for Group and Individual Fairness in Federated Learning , 2021, ArXiv.

[293]  Sergio Grammatico,et al.  Distributed Demand Side Management With Stochastic Wind Power Forecasting , 2022, IEEE Transactions on Control Systems Technology.

[294]  G. Laporte,et al.  The diala-ride problem : models and algorithms , .