Generating heuristics for novice players

We consider the problem of generating compact sub-optimal game-playing heuristics that can be understood and easily executed by novices. In particular, we seek to find heuristics that can lead to good play while at the same time be expressed as fast and frugal trees or short decision lists. This has applications in automatically generating tutorials and instructions for playing games, but also in analyzing game design and measuring game depth. We use the classic game Blackjack as a testbed, and compare condition induction with the RIPPER algorithm, exhaustive-greedy search in statement space, genetic programming and axis-aligned search. We find that all of these methods can find compact well-playing heuristics under the given constraints, with axis-aligned search performing particularly well.

[1]  Steven J. Brams,et al.  Catch-Up: A Game in Which the Lead Alternates , 2016 .

[2]  David B. Fogel.  Evolving strategies in blackjack , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[3]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[4]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[5]  Riccardo Poli,et al.  Evolutionary Solo Pong players , 2005, 2005 IEEE Congress on Evolutionary Computation.

[6]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[7]  E. Sanchez,et al.  Blackjack as a test bed for learning strategies in neural networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[8]  Michael Buro,et al.  From Simple Features to Sophisticated Evaluation Functions , 1998, Computers and Games.

[9]  Gerd Gigerenzer,et al.  Fast and frugal heuristics: The tools of bounded rationality , 2004 .

[10]  Julian Togelius,et al.  Evolving controllers for simulated car racing using object oriented genetic programming , 2007, GECCO '07.

[11]  Jonathan Schaeffer,et al.  A World Championship Caliber Checkers Program , 1992, Artif. Intell..

[12]  Moshe Sipper,et al.  Evolving Lose-Checkers players using genetic programming , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[13]  Julian Togelius,et al.  Super mario evolution , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[14]  D. Kahneman.  Maps of Bounded Rationality: Psychology for Behavioral Economics , 2003 .

[15]  Moshe Sipper,et al.  GP-Gammon: Genetically Programming Backgammon Players , 2005, Genetic Programming and Evolvable Machines.

[16]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[17]  Graham Kendall,et al.  The evolution of blackjack strategies , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[18]  Luigi Barone,et al.  An adaptive learning model for simplified poker using evolutionary algorithms , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[19]  George Skaff Elias,et al.  Characteristics of Games , 2012 .

[20]  Moshe Sipper,et al.  GP-EndChess: Using Genetic Programming to Evolve Chess Endgame Players , 2005, EuroGP.

[21]  L. Humble,et al.  The World's Greatest Blackjack Book , 1980 .

[22]  Frédéric Maire,et al.  Evolutionary Game Design , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[23]  James Paul Gee,et al.  Learning by Design: Good Video Games as Learning Machines , 2005 .

[24]  Marvin R. G. Schiller,et al.  A Comparison between Cognitive and AI Models of Blackjack Strategy Learning , 2012, KI.

[25]  Marc Parizeau,et al.  DEAP: a python framework for evolutionary algorithms , 2012, GECCO '12.

[26]  William Bart,et al.  Moves in Mind: The Psychology of Board Games , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[27]  Moshe Sipper,et al.  GP-Robocode: Using Genetic Programming to Evolve Robocode Players , 2005, EuroGP.

[28]  Ronald L. Rivest,et al.  Learning decision lists , 2004, Machine Learning.

[29]  Edward O. Thorp,et al.  Beat the Dealer: A Winning Strategy for the Game of Twenty-One , 1965 .

[30]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[31]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[32]  R. R. Baldwin,et al.  The Optimum Strategy in Blackjack , 1956 .

[33]  Barkley Vowk,et al.  An Evolutionary Approach Generates Human Competitive Corewar Programs , 2004 .

[34]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[35]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.