Federated Learning with Fair Worker Selection: A Multi-Round Submodular Maximization Approach

In this paper, we study the problem of fair worker selection in Federated Learning systems, where fairness serves as an incentive mechanism that encourages more workers to participate in the federation. Considering the achieved training accuracy of the global model as the utility of the selected workers, which is typically a monotone submodular function, we formulate the worker selection problem as a new multi-round monotone submodular maximization problem with cardinality and fairness constraints. The objective is to maximize the time-average utility over multiple rounds subject to an additional fairness requirement that each worker must be selected for a certain fraction of time. While the traditional submodular maximization with a cardinality constraint is already a well-known NP-Hard problem, the fairness constraint in the multi-round setting adds an extra layer of difficulty. To address this novel challenge, we propose three algorithms: Fair Continuous Greedy (FairCGl and FairCG2) and Fair Discrete Greedy (FairDG), all of which satisfy the fairness requirement whenever feasible. Moreover, we prove nontrivial lower bounds on the achieved time-average utility under FairCGl and FairCG2. In addition, by giving a higher priority to fairness, FairDG ensures a stronger short-term fairness guarantee, which holds in every round. Finally, we perform extensive simulations to verify the effectiveness of the proposed algorithms in terms of the time-average utility and fairness satisfaction.

[1]  Tian Li,et al.  Fair Resource Allocation in Federated Learning , 2019, ICLR.

[2]  Jia Liu,et al.  Combinatorial Sleeping Bandits with Fairness Constraints , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[3]  Jianxin Wu,et al.  GROPING: Geomagnetism and cROwdsensing Powered Indoor NaviGation , 2015, IEEE Transactions on Mobile Computing.

[4]  Stephen L. Smith,et al.  Submodularity and greedy algorithms in sensor scheduling for linear dynamical systems , 2015, Autom..

[5]  Ying-Chang Liang,et al.  Federated Learning in Mobile Edge Networks: A Comprehensive Survey , 2020, IEEE Communications Surveys & Tutorials.

[6]  Vasileios Tzoumas,et al.  Resilient Submodular Maximization for Control and Sensing , 2018 .

[7]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[8]  Reynold Cheng,et al.  Online Influence Maximization , 2015, KDD.

[9]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[10]  Maxim Sviridenko,et al.  Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee , 2004, J. Comb. Optim..

[11]  Lin Gao,et al.  Providing long-term participation incentive in participatory sensing , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[12]  Qing Zeng-Treitler,et al.  Predicting sample size required for classification performance , 2012, BMC Medical Informatics and Decision Making.

[13]  Bingsheng He,et al.  A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection , 2019, IEEE Transactions on Knowledge and Data Engineering.

[14]  Rajiv Gandhi,et al.  Dependent rounding and its applications to approximation algorithms , 2006, JACM.

[15]  A. An,et al.  Scalable Deletion-Robust Submodular Maximization: Data Summarization with Privacy and Fairness Constraints , 2018 .

[16]  Jan Vondrák,et al.  Maximizing a Submodular Set Function Subject to a Matroid Constraint (Extended Abstract) , 2007, IPCO.

[17]  Alexandra Chouldechova,et al.  The Frontiers of Fairness in Machine Learning , 2018, ArXiv.

[18]  Jeff A. Bilmes,et al.  Submodularity for Data Selection in Machine Translation , 2014, EMNLP.

[19]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[20]  Andreas Krause,et al.  Submodular Function Maximization , 2014, Tractability.

[21]  Mehryar Mohri,et al.  Agnostic Federated Learning , 2019, ICML.

[22]  H. Vincent Poor,et al.  Scheduling Policies for Federated Learning in Wireless Networks , 2019, IEEE Transactions on Communications.

[23]  Y. Narahari,et al.  Stochastic Multi-armed Bandits with Arm-specific Fairness Guarantees , 2019, ArXiv.

[24]  Jan Vondrák,et al.  Randomized Pipage Rounding for Matroid Polytopes and Applications , 2009, ArXiv.

[25]  Rishabh K. Iyer,et al.  Submodularity in Data Subset Selection and Active Learning , 2015, ICML.

[26]  Andreas Krause,et al.  Streaming submodular maximization: massive data summarization on the fly , 2014, KDD.

[27]  Jan Vondrák,et al.  Optimal approximation for the submodular welfare problem in the value oracle model , 2008, STOC.

[28]  J. Bilmes,et al.  Diverse Client Selection for Federated Learning:Submodularity and Convergence Analysis , 2021 .

[29]  Cyrus Shahabi,et al.  A Server-Assigned Spatial Crowdsourcing Framework , 2015, ACM Trans. Spatial Algorithms Syst..

[30]  Niv Buchbinder,et al.  Submodular Functions Maximization Problems , 2018, Handbook of Approximation Algorithms and Metaheuristics.

[31]  Philip S. Yu,et al.  Multi-Round Influence Maximization , 2018, KDD.

[32]  Bo Thiesson,et al.  The Learning-Curve Sampling Method Applied to Model-Based Clustering , 2002, J. Mach. Learn. Res..

[33]  Jan Vondrák,et al.  Dependent Randomized Rounding via Exchange Properties of Combinatorial Structures , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[34]  Haris Vikalo,et al.  Greedy sensor selection: Leveraging submodularity , 2010, 49th IEEE Conference on Decision and Control (CDC).

[35]  Takayuki Nishio,et al.  Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[36]  Richard M. Murray,et al.  On a stochastic sensor selection algorithm with applications in sensor scheduling and sensor coverage , 2006, Autom..

[37]  Ning Zhang,et al.  Identifying the Most Valuable Workers in Fog-Assisted Spatial Crowdsourcing , 2017, IEEE Internet of Things Journal.

[38]  Jan Vondrák,et al.  Fast algorithms for maximizing submodular functions , 2014, SODA.