Elo-MMR: A Rating System for Massive Multiplayer Competitions

Skill estimation mechanisms, colloquially known as rating systems, play an important role in competitive sports and games. They provide a measure of player skill, which incentivizes competitive performances and enables balanced match-ups. In this paper, we present a novel Bayesian rating system for contests with many participants. It is widely applicable to competition formats with discrete ranked matches, such as online programming competitions, obstacle courses races, and video games. The system’s simplicity allows us to prove theoretical bounds on its robustness and runtime. In addition, we show that it is incentive-compatible: a player who seeks to maximize their rating will never want to underperform. Experimentally, the rating system surpasses existing systems in prediction accuracy, and computes faster than existing systems by up to an order of magnitude.

[1]  Chih-Jen Lin,et al.  Ranking individuals by group comparisons , 2006, ICML.

[2]  Chih-Jen Lin,et al.  A Bayesian Approximation Method for Online Ranking , 2011, J. Mach. Learn. Res..

[3]  Christopher M. Bishop,et al.  Model-based machine learning , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[4]  M. An.  Log-Concave Probability Distributions: Theory and Statistical Testing , 1996 .

[5]  Yao Li,et al.  Learning from Group Comparisons : Exploiting Higher Order Interactions , 2018 .

[6]  Yordan Zaykov,et al.  TrueSkill 2: An improved Bayesian skill rating system , 2018 .

[7]  G. Vuckovic,et al.  Reliability of Judging in DanceSport , 2019, Front. Psychol..

[8]  Rémi Coulom,et al.  Whole-History Rating: A Bayesian Rating System for Players of Time-Varying Strength , 2008, Computers and Games.

[9]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[10]  Thorsten Joachims,et al.  Modeling Intransitivity in Matchup and Comparison Data , 2016, WSDM.

[11]  D. Freedman.  On the Asymptotic Behavior of Bayes' Estimates in the Discrete Case , 1963 .

[12]  B. Mantin,et al.  Forecasting sales of new virtual goods with the Elo rating system , 2014 .

[13]  Tom Minka,et al.  TrueSkill Through Time: Revisiting the History of Chess , 2007, NIPS.

[14]  M. Glickman.  Parameter Estimation in Large Dynamic Paired Comparison Experiments , 1999 .

[15]  Tom Minka,et al.  TrueSkillTM: A Bayesian Skill Rating System , 2006, NIPS.

[16]  Jacob R. Lorch,et al.  Matchmaking for online games and other latency-sensitive P2P systems , 2009, SIGCOMM '09.

[17]  Stephanie Kovalchik,et al.  Extension of the Elo rating system to margin of victory , 2020 .

[18]  Changjie Fan,et al.  OptMatch: Optimized Matchmaking via Modeling the High-Order Interactions on the Arena , 2020, KDD.