DeceFL: A Principled Decentralized Federated Learning Framework

Traditional machine learning relies on a centralized data pipeline, i.e., data are provided to a central server for model training. In many applications, however, data are inherently fragmented. Such a decentralized nature of these databases presents the biggest challenge for collaboration: sending all decentralized datasets to a central server raises serious privacy concerns. Although there has been a joint effort in tackling such a critical issue by proposing privacy-preserving machine learning frameworks, such as federated learning, most state-of-the-art frameworks are built still in a centralized way, in which a central client is needed for collecting and distributing model information (instead of data itself) from every other client, leading to high communication pressure and high vulnerability when there exists a failure at or attack on the central client. Here we propose a principled decentralized federated learning algorithm (DeceFL), which does not require a central client and relies only on local information transmission between clients and their neighbors, representing a fully decentralized learning framework. It has been further proven that every client reaches the global minimum with zero performance gap and achieves the same convergence rate O(1/T) (where T is the number of iterations in gradient descent) as centralized federated learning when the loss function is smooth and strongly convex. Finally, the proposed algorithm has been applied to a number of applications to illustrate its effectiveness for both convex and nonconvex loss functions, demonstrating its applicability to a wide range of real-world medical and industrial applications. 1School of Artificial Intelligence and Automation, Huazhong University of Science and Technology. 2School of Mechanical Science and Engineering, Huazhong University of Science and Technology. 3Department of Applied Mathematics, University of Waterloo. 4State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University. 5School of Electrical Engineering and Computer Science, and Digital Futures, KTH Royal Institute of Technology. 6AVIC Chengdu Aircraft Industrial (Group) Co., Ltd.. ∗Equal contributions. Email: yye@hust.edu.cn. November 2, 2021 DRAFT ar X iv :2 10 7. 07 17 1v 2 [ cs .L G ] 3 0 O ct 2 02 1