Using Heuristic-Search Based Engines for Estimating Human Skill at Chess

Establishing heuristic-search based chess programs as appropriate tools for estimating human skill levels at chess may seem impossible due to the following issues: the programs’ evaluations and decisions tend to change with the depth of search and with the program used. In this research, we provide an analysis of the differences between heuristic-search based programs in estimating chess skill. We used four different chess programs to perform analyses of large data sets of recorded human decisions, and obtained very similar rankings of skill-based performances of selected chess players using any of these programs at various levels of search. A conclusion is that, given two chess players, all the programs unanimously rank one player to be clearly stronger than the other, or all the programs assess their strengths to be similar. We also repeated our earlier analysis with the program CRAFTY of World Chess Champions with currently one of the strongest chess programs, RYBKA 32, and obtained qualitatively very similar results as with CRAFTY. This speaks in favour of computer heuristic search being adequate for estimating skill levels of chess players, despite the above stated issues.

[1]  Ivan Bratko,et al.  Factors Affecting Diminishing Returns for Searching Deeper , 2007, J. Int. Comput. Games Assoc..

[2]  Matej Guid.  Search and Knowledge for Human and Machine Problem Solving , 2010 .

[3]  J. Renze Steenhuizen.  New Results in Deep-Search Behaviour , 2005, J. Int. Comput. Games Assoc..

[4]  Giuseppe Di Fatta,et al.  Skill rating by Bayesian inference , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[5]  Ivan Bratko,et al.  Computer Analysis of World Chess Champions , 2006, J. Int. Comput. Games Assoc..

[6]  Ken Thompson,et al.  Retrograde Analysis of Certain Endgames , 1986, J. Int. Comput. Games Assoc..

[7]  Guy Haworth.  Gentlemen, Stop Your Engines! , 2007, J. Int. Comput. Games Assoc..

[8]  Giuseppe Di Fatta,et al.  Performance and Prediction: Bayesian Modelling of Fallible Choice in Chess , 2009, ACG.

[9]  Göran Grottling,et al.  The Swedish Rating List , 1996, J. Int. Comput. Games Assoc..

[10]  Ivan Bratko,et al.  HOW TRUSTWORTHY IS CRAFTY’S ANALYSIS OF WORLD CHESS CHAMPIONS? , 2008 .