Rating systems with multiple factors

Rating systems have been receiving increasing attention recently, especially after TrueSkill was introduced (Herbrich et al., 2007). Most existing models are based upon one latent variable associated with each player; the purpose of my project is to construct a multiple-feature model for rating players. Such a model associates more characteristics to a competitor and could – besides telling your skill and being used for matching players – provide insight into the characteristics of one’s play and strategy. We found that simply fitting the models through maximum likelihood has low generalising capacity, and also requires massive amounts of data in order to yield high accuracy. We turned towards a Bayesian approach and used Assumed density filtering and Expectation Propagation algorithms (Minka, 2001). They bring a significant accuracy bonus, even without a time series model to keep track of how players’ skills evolve. We have also implemented a version of TrueSkill adapted to our problem (game of Go) and use it for comparing our models. We present experimental evidence on the increased performance of the multiple factors models; they significantly raise the accuracy of Expectation Propagation model, and with enough data, more factors improve also the Assumed density filtering model. On small datasets, we discovered that an iterative method brings a significant advantage for Assumed density filtering, greatly surpassing even the TrueSkill algorithm. There are some additional benefits for the multiple factors approach, including higher accuracy for predicting results of balanced games.