Limits of PageRank-based ranking methods in sports data

While PageRank has been extensively used to rank sport tournament participants (teams or individuals), its superiority over simpler ranking methods has been never clearly demonstrated. We use sports results from 18 major leagues to calibrate a state-of-art model for synthetic sports results. Model data are then used to assess the ranking performance of PageRank in a controlled setting. We find that PageRank outperforms the benchmark ranking by the number of wins only when a small fraction of all games have been played. Increased randomness in the data, such as intrinsic randomness of outcomes or advantage of home teams, further reduces the range of PageRank's superiority. We propose a new PageRank variant which outperforms PageRank in all evaluated settings, yet shares its sensitivity to increased randomness in the data. Our main findings are confirmed by evaluating the ranking algorithms on real data. Our work demonstrates the danger of using novel metrics and algorithms without considering their limits of applicability.

[1]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[2]  Matthieu Lerasle,et al.  The number of potential winners in Bradley-Terry model in random environment , 2015, 1509.07265.

[3]  Linyuan Lu,et al.  Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data , 2020, J. Informetrics.

[4]  R A Bradley.  Science, statistics, and paired comparisons. , 1976, Biometrics.

[5]  Haroldo V. Ribeiro,et al.  The Advantage of Playing Home in NBA: Microscopic, Team-Specific and Evolving Features , 2016, PloS one.

[6]  Richard Giulianotti,et al.  Sport: A Critical Sociology , 2004 .

[7]  J. Curtis,et al.  The Social Significance of Sport: An Introduction to the Sociology of Sport , 1989 .

[8]  Naoki Masuda,et al.  A network-based dynamical ranking system for competitive sports , 2012, Scientific Reports.

[9]  Weibing Deng,et al.  Universal scaling in sports ranking , 2011, 1111.2919.

[10]  Mathias Pessiglione,et al.  Imaging Social Motivation: Distinct Brain Mechanisms Drive Effort Production during Collaboration versus Competition , 2013, The Journal of Neuroscience.

[11]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS , 1952 .

[12]  Aric Hagberg,et al.  Exploring Network Structure, Dynamics, and Function using NetworkX , 2008, Proceedings of the Python in Science Conference.

[13]  M. Newman,et al.  A network-based ranking system for US college football , 2005, physics/0505169.

[14]  Gourab Ghoshal,et al.  Ranking stability and super-stable nodes in complex networks. , 2011, Nature communications.

[15]  Da Silva Sergio,et al.  Hidden power law patterns in the top European football leagues , 2013 .

[16]  F. Vazquez,et al.  Randomness in Competitions , 2012, Journal of Statistical Physics.

[17]  Filippo Radicchi,et al.  Who Is the Best Player Ever? A Complex Network Analysis of the History of Professional Tennis , 2011, PloS one.

[18]  Lasko Basnarkov,et al.  PageRank Approach to Ranking National Football Teams , 2015, ArXiv.

[19]  Linyuan Lu,et al.  Link Prediction in Complex Networks: A Survey , 2010, ArXiv.

[20]  Scott R. Rosner,et al.  The Business of Sports , 2004 .

[21]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[22]  Manuela Cattelan,et al.  Models for Paired Comparison Data: A Review with Emphasis on Dependent Data , 2012, 1210.1016.

[23]  Nigel J. Balmer,et al.  The extent and causes of home advantage: Some recent insights , 2005 .

[24]  P. V. Rao,et al.  Ties in Paired-Comparison Experiments: A Generalization of the Bradley-Terry Model , 1967 .

[25]  Yi-Cheng Zhang,et al.  Ranking nodes in growing networks: When PageRank fails , 2015, Scientific Reports.

[26]  Marcos André Gonçalves,et al.  Time-Aware Ranking in Sport Social Networks , 2012, J. Inf. Data Manag..

[27]  Charles Redmond,et al.  A Natural Generalization of the Win-Loss Rating System , 2003 .

[28]  A. Spanias,et al.  Tennis Player Ranking using Quantitative Models , 2013 .

[29]  C. D. Meyer,et al.  Generalizing Google ’ s PageRank to Rank National Football League Teams , 2008 .

[30]  Renato Assunção,et al.  Luck is Hard to Beat: The Difficulty of Sports Prediction , 2017, KDD.

[31]  László Csató,et al.  An application of incomplete pairwise comparison matrices for ranking top tennis players , 2014, Eur. J. Oper. Res..

[32]  P. Moran.  On the method of paired comparisons. , 1947, Biometrika.

[33]  Christina M. Frederick-Recascino,et al.  Competition and Intrinsic Motivation in Physical Activity: A Comparison of Two Groups , 2003 .

[34]  Satyam Mukherjee.  Identifying the greatest team and captain—A complex network approach to cricket matches , 2012 .

[35]  Amy Nicole Langville,et al.  Sensitivity and Stability of Ranking Vectors , 2011, SIAM J. Sci. Comput..

[36]  John Cairns,et al.  The Economic Design of Sporting Contests , 2003 .

[37]  Phillip Hone,et al.  Measuring the Contribution of Sport to the Economy , 2006 .

[38]  James P. Keener,et al.  The Perron-Frobenius Theorem and the Ranking of Football Teams , 1993, SIAM Rev..

[39]  Nils Lid Hjort,et al.  Model Selection and Model Averaging , 2001 .

[40]  David F. Gleich,et al.  PageRank beyond the Web , 2014, SIAM Rev..

[41]  Raphael Flepp,et al.  Dealing With Randomness in Match Outcomes: How to Rethink Performance Evaluation in European Club Football Using Expected Goals , 2020 .

[42]  Roman M. Sheremeta,et al.  A survey of experimental research on contests, all-pay auctions and tournaments , 2012, Experimental Economics.

[43]  O'Malley A. James.  Probability Formulas and Statistical Analysis in Tennis , 2008 .

[44]  Neil Tranter,et al.  Sport, Economy and Society in Britain 1750-1914 , 1998 .