On Classification of Games and Evaluation of Players - with Some Sweeping Generalizations About the Literature

In the literature of machine learning in games, we sense that while certain aspects of machine learning for game-playing purposes have been well covered, other aspects have been inadequately treated or even ignored. In particular, we see two subjects as requiring more attention: the study of imperfect-information games and the choice of evaluation methods for game-playing agents. We argue that theoretical efforts and awareness are needed in these areas, and, as a contribution to the latter of these subjects, present a set of several evaluation criteria for game-playing agents. These include criteria which 1) calculate the performance of agents when playing against certain actual or hypothetical opponents, 2) measure the similarity of strategies to game solutions, and 3) calculate the abilit y of agents to estimate the advantages or disadvantages of game states and actions. The applicabilit y and usefulness of these criteria for different situations and purposes is also discussed.