On the Strategyproofness of the Geometric Median

The geometric median, an instrumental component of the secure machine learning toolbox, is known to be effective when robustly aggregating models (or gradients), gathered from potentially malicious (or strategic) users. What is less known is the extent to which the geometric median incentivizes dishonest behaviors. This paper addresses this fundamental question by quantifying its strategyproofness. While we observe that the geometric median is not even approximately strategyproof, we prove that it is asymptotically $\alpha$-strategyproof: when the number of users is large enough, a user that misbehaves can gain at most a multiplicative factor $\alpha$, which we compute as a function of the distribution followed by the users. We then generalize our results to the case where users actually care more about specific dimensions, determining how this impacts $\alpha$. We also show how the skewed geometric medians can be used to improve strategyproofness.

[1]  R. Guerraoui,et al.  Byzantine Machine Learning Made Easy by Resilient Averaging of Momentums , 2022, ICML.

[2]  R. Guerraoui,et al.  An Equivalence Between Data Poisoning and Byzantine Gradient Attacks , 2022, ICML.

[3]  L. Hoang,et al.  Robust Sparse Voting , 2022, ArXiv.

[4]  Ufuk Topcu,et al.  Robust Training in High Dimensions via Block Coordinate Geometric Median Descent , 2021, AISTATS.

[5]  R. Guerraoui,et al.  Strategyproof Learning: Building Trustworthy User-Generated Datasets , 2021, ArXiv.

[6]  Yuexiang Yang,et al.  Detecting Malicious Model Updates from Federated Learning on Conditional Variational Autoencoder , 2021, 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[7]  Inderpal Singh,et al.  The hype machine: How social media disrupts our elections, our economy and our health‐ and how we must adapt , 2021 .

[8]  Lê Nguyên Hoang,et al.  Science Communication Desperately Needs More Aligned Recommendation Algorithms , 2020, Frontiers in Communication.

[9]  Gianluca Stringhini,et al.  Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incels , 2020, Proc. ACM Hum. Comput. Interact..

[10]  Filip Hanzely,et al.  Lower Bounds and Optimal Algorithms for Personalized Federated Learning , 2020, NeurIPS.

[11]  Toby Walsh,et al.  Strategy Proof Mechanisms for Facility Location in Euclidean and Manhattan Space , 2020, ArXiv.

[12]  A. Salman Avestimehr,et al.  Byzantine-Resilient Secure Federated Learning , 2020, IEEE Journal on Selected Areas in Communications.

[13]  Sumit Goel,et al.  Coordinate-wise Median: Not Bad, Not Bad, Pretty Good , 2020, ArXiv.

[14]  W. Soral,et al.  Hate Speech Epidemic. The Dynamic Effects of Derogatory Language on Intergroup Relations and Political Radicalization , 2020, Political Psychology.

[15]  Sai Praneeth Karimireddy,et al.  Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing , 2020, ICLR.

[16]  Nguyen H. Tran,et al.  Personalized Federated Learning with Moreau Envelopes , 2020, NeurIPS.

[17]  P. Mena Cleaning Up Social Media: The Effect of Warning Labels on Likelihood of Sharing False News on Facebook , 2020, Policy & Internet.

[18]  Jenifer Whitten-Woodring,et al.  Poison If You Don’t Know How to Use It: Facebook, Democracy, and Human Rights in Myanmar , 2020 .

[19]  Mor Naaman,et al.  Characterizing Twitter Users Who Engage in Adversarial Interactions against Political Candidates , 2020, CHI.

[20]  Shujhat Khan,et al.  Coronavirus: the spread of misinformation , 2020, BMC Medicine.

[21]  Zaïd Harchaoui,et al.  Robust Aggregation for Federated Learning , 2019, IEEE Transactions on Signal Processing.

[22]  Qing Ling,et al.  Federated Variance-Reduced Stochastic Gradient Descent With Robustness to Byzantine Attacks , 2019, IEEE Transactions on Signal Processing.

[23]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2019, Found. Trends Mach. Learn..

[24]  Ariel D. Procaccia,et al.  WeBuildAI , 2019, Proc. ACM Hum. Comput. Interact..

[25]  Carmine Ventre,et al.  Mechanism Design for Constrained Heterogeneous Facility Location , 2019, SAGT.

[26]  Hongyi Wang,et al.  DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation , 2019, NeurIPS.

[27]  Dingli Yu,et al.  Characterization of Group-strategyproof Mechanisms for Facility Location in Strictly Convex Space , 2018, EC.

[28]  Vincent Conitzer,et al.  Adapting a Kidney Exchange Algorithm to Align with Human Values , 2018, AAAI.

[29]  Dan Alistarh,et al.  Byzantine Stochastic Gradient Descent , 2018, NeurIPS.

[30]  Rachid Guerraoui,et al.  The Hidden Vulnerability of Distributed Learning in Byzantium , 2018, ICML.

[31]  Rachid Guerraoui,et al.  Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent , 2017, NIPS.

[32]  Iyad Rahwan,et al.  A Voting-Based System for Ethical Decision Making , 2017, AAAI.

[33]  Lili Su,et al.  Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent , 2019, PERV.

[34]  Lê Nguyên Hoang,et al.  Strategy-proofness of the randomized Condorcet voting system , 2017, Soc. Choice Welf..

[35]  Jakub W. Pachocki,et al.  Geometric median in nearly linear time , 2016, STOC.

[36]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[37]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[38]  Tilman Börgers,et al.  An introduction to the theory of mechanism design , 2015 .

[39]  Ufuk Topcu,et al.  An approximately truthful mechanism for electric vehicle charging via joint differential privacy , 2015, 2015 American Control Conference (ACC).

[40]  Song Guo,et al.  Approximately Truthful Mechanisms for Radio Spectrum Allocation , 2015, IEEE Transactions on Vehicular Technology.

[41]  Craig Boutilier,et al.  Approximately Strategy-proof Mechanisms for (Constrained) Facility Location , 2015, AAMAS.

[42]  Luc Racaut Vox Populi: Une Histoire du Vote avant le Suffrage Universel , 2015 .

[43]  Jay Sethuraman,et al.  Strategyproof Mechanisms for One-Dimensional Hybrid and Obnoxious Facility Location Models , 2014, AAAI Workshop: Incentive and Trust in E-Communities.

[44]  Qiang Zhang,et al.  Strategyproof mechanism design for facility location games with weighted agents on a line , 2014, J. Comb. Optim..

[45]  S. Matthew Weinberg,et al.  Bayesian Truthful Mechanisms for Job Scheduling from Bi-criterion Approximation Algorithms , 2014, SODA.

[46]  D. Murphey,et al.  The Righteous Mind: Why Good People Are Divided by Politics and Religion , 2013 .

[47]  Stanislav Minsker Geometric median and robust estimation in Banach spaces , 2013, 1308.1334.

[48]  Yang Cai,et al.  Understanding Incentives: Mechanism Design Becomes Algorithm Design , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[49]  Yang Cai,et al.  Reducing Revenue to Welfare Maximization: Approximation Algorithms and other Generalizations , 2013, SODA.

[50]  Yang Cai,et al.  Optimal Multi-dimensional Mechanism Design: Reducing Revenue to Welfare Maximization , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[51]  Christos Tzamos,et al.  On the Power of Deterministic Mechanisms for Facility Location Games , 2012, TEAC.

[52]  Yang Cai,et al.  An algorithmic characterization of multi-dimensional mechanisms , 2011, STOC '12.

[53]  Olivier Spanjaard,et al.  Strategy-Proof Mechanisms for Facility Location Games with Many Facilities , 2011, ADT.

[54]  M. Balinski,et al.  Majority Judgment: Measuring, Ranking, and Electing , 2011 .

[55]  Alexander J. Smola,et al.  Parallelized Stochastic Gradient Descent , 2010, NIPS.

[56]  Yuan Zhou,et al.  Tighter Bounds for Facility Games , 2009, WINE.

[57]  Noga Alon,et al.  Strategyproof Approximation Mechanisms for Location on Networks , 2009, ArXiv.

[58]  Moshe Tennenholtz,et al.  Approximate mechanism design without money , 2009, EC '09.

[59]  Jeffrey C. Ely,et al.  Foundations of Dominant Strategy Mechanisms , 2007 .

[60]  Salvador Barberà,et al.  Strategy-proof voting on compact ranges , 1998 .

[61]  S. Barberà,et al.  Voting under Constraints , 1997 .

[62]  Jack Brimberg,et al.  The Fermat—Weber location problem revisited , 1995, Math. Program..

[63]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[64]  H. P. Lopuhaä On the relation between S-estimators and M-estimators of multivariate location and covariance , 1989 .

[65]  David J. Smith,et al.  How Small Is a Unit Ball , 1989 .

[66]  Pierre Hansen,et al.  The Minisum and Minimax Location Problems Revisited , 1985, Oper. Res..

[67]  Fred W. Roush,et al.  Nonmanipulability in two dimensions , 1984 .

[68]  Roger B. Myerson,et al.  Optimal Auction Design , 1981, Math. Oper. Res..

[69]  M. Satterthwaite,et al.  Strategy-proofness and single-peakedness , 1976 .

[70]  M. Satterthwaite Strategy-proofness and Arrow's conditions: Existence and correspondence theorems for voting procedures and social welfare functions , 1975 .

[71]  Theodore Groves,et al.  Incentives in Teams , 1973 .

[72]  A. Gibbard Manipulation of Voting Schemes: A General Result , 1973 .

[73]  E. H. Clarke Multipart pricing of public goods , 1971 .

[74]  William Vickrey,et al.  Counterspeculation, Auctions, And Competitive Sealed Tenders , 1961 .

[75]  K. Arrow A Difficulty in the Concept of Social Welfare , 1950, Journal of Political Economy.

[76]  L. Hoang,et al.  Recommendation Algorithms, a Neglected Opportunity for Public Health REVUE MÉDECINE ET PHILOSOPHIE , 2021 .

[77]  Ronan Le Bras,et al.  Delphi: Towards Machine Ethics and Norms , 2021, ArXiv.

[78]  Prashanth Bhat,et al.  Covert Hate Speech: White Nationalists and Dog Whistle Communication on Twitter , 2020 .

[79]  Richard Lee Brady,et al.  A spatial analogue of May’s Theorem , 2017, Soc. Choice Welf..

[80]  Yang Cai,et al.  Reducing Bayesian Mechanism Design to Algorithm Design , 2016, Encyclopedia of Algorithms.

[81]  Lawrence G. Sager Handbook of Computational Social Choice , 2015 .

[82]  David C. Parkes,et al.  Approximate Strategyproofness , 2012 .

[83]  George O. Wesolowsky,et al.  THE WEBER PROBLEM: HISTORY AND PERSPECTIVES. , 1993 .

[84]  Kim C. Border,et al.  Straightforward Elections, Unanimity, and Phantom Voters , 1983 .