Search, Inference and Opponent Modelling in an Expert-Caliber Skat Player

In this dissertation, we investigate the problems of search, inference and opponent modelling in imperfect information games through the creation of a computer player for the popular german card game skat. In so doing, we demonstrate three major contributions to the field of artificial intelligence research in games. First, we present our skat player Kermit which, using a synthesis of different techniques, decisively defeats previously existing computer players and displays playing strength comparable to human experts. Second, we propose a framework for evaluating game-playing algorithms with known theoretical flaws and explaining the success of such methods in different classes of games. Finally, we enhance Kermit with a simple but effective opponent modelling component that allows it to adapt and improve its performance against players of differing playing strength in real time.

[1]  Thomas Keller,et al.  Automatic Bidding for the Game of Skat , 2008, KI.

[2]  Duane Szafron,et al.  Using counterfactual regret minimization to create competitive multiplayer poker agents , 2010, AAMAS 2010.

[3]  C. Shannon.  A chess-playing machine. , 1950, Scientific American.

[4]  Nathan R. Sturtevant,et al.  Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search , 2010, AAAI.

[5]  Dana S. Nau,et al.  Computer Bridge - A Big Win for AI Planning , 1998, AI Mag..

[6]  Michael H. Bowling,et al.  Data Biased Robust Counter Strategies , 2009, AISTATS.

[7]  Alan Bundy,et al.  Combining Knowledge and Search to Solve Single-Suit Bridge , 2000, AAAI/IAAI.

[8]  Sylvain Gelly,et al.  Modifications of UCT and sequence-like simulations for Monte-Carlo Go , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[9]  Nathan R. Sturtevant,et al.  An Analysis of UCT in Multi-Player Games , 2008, J. Int. Comput. Games Assoc..

[10]  Michael Buro,et al.  From Simple Features to Sophisticated Evaluation Functions , 1998, Computers and Games.

[11]  Tuomas Sandholm,et al.  Game theory-based opponent modeling in large imperfect-information games , 2011, AAMAS.

[12]  Jacek Mandziuk,et al.  Learning Without Human Expertise: A Case Study of the Double Dummy Bridge Problem , 2009, IEEE Transactions on Neural Networks.

[13]  Michael Buro,et al.  Using Payoff-Similarity to Speed Up Search , 2011, IJCAI.

[14]  Tuomas Sandholm,et al.  Finding equilibria in large sequential games of imperfect information , 2006, EC '06.

[15]  Michael Buro,et al.  Minimum Proof Graphs and Fastest-Cut-First Search Heuristics , 2009, IJCAI.

[16]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Jonathan Schaeffer,et al.  Improved Opponent Modeling in Poker , 2000 .

[18]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[19]  Nathan R. Sturtevant,et al.  Feature Construction for Reinforcement Learning in Hearts , 2006, Computers and Games.

[20]  Michael H. Bowling,et al.  Bayes' Bluff: Opponent Modelling in Poker , 2005, UAI 2005.

[21]  Jonathan Schaeffer,et al.  The challenge of poker , 2002, Artif. Intell..

[22]  Elizabeth A. Kemp,et al.  Issues in Designing Tutors for Games of Incomplete Information: a Bridge Case Study , 2005, SGAI Conf..

[23]  Matthew L. Ginsberg,et al.  GIB: Imperfect Information in a Computationally Challenging Game , 2011, J. Artif. Intell. Res..

[24]  V. S. Subrahmanian,et al.  Game-Tree Search with Combinatorially Large Belief States , 2005, IJCAI.

[25]  Gerald Tesauro,et al.  TD-Gammon: A Self-Teaching Backgammon Program , 1995 .

[26]  Gerald Tesauro,et al.  TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play , 1994, Neural Computation.

[27]  Ian Frank,et al.  Search in Games with Incomplete Information: A Case Study Using Bridge Card Play , 1998, Artificial Intelligence.

[28]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[29]  Jeff Yan,et al.  Collusion Detection in Online Bridge , 2010, AAAI.

[30]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[31]  Michael H. Bowling,et al.  Computing Robust Counter-Strategies , 2007, NIPS.

[32]  Yngvi Björnsson,et al.  Simulation-Based Approach to General Game Playing , 2008, AAAI.

[33]  C. E. Lemke,et al.  Equilibrium Points of Bimatrix Games , 1964 .

[34]  Bernhard von Stengel,et al.  Fast algorithms for finding randomized strategies in game trees , 1994, STOC '94.

[35]  Nathan R. Sturtevant,et al.  Current Challenges in Multi-player Game Search , 2004, Computers and Games.

[36]  Malte Helmert,et al.  A Skat Player Based on Monte-Carlo Simulation , 2006, Computers and Games.

[37]  Paolo Ciancarini,et al.  Representing Kriegspiel States with Metapositions , 2007, IJCAI.

[38]  Mark Richards,et al.  Opponent Modeling in Scrabble , 2007, IJCAI.

[39]  Jonathan Schaeffer,et al.  CHINOOK: The World Man-Machine Checkers Champion , 1996, AI Mag..

[40]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[41]  Shaul Markovitch,et al.  Learning to bid in bridge , 2006, Machine Learning.

[42]  Michael Buro,et al.  Real-Time Opponent Modeling in Trick-Taking Card Games , 2011, IJCAI.

[43]  Jean Méhat,et al.  Ary, a general game playing program , 2010 .

[44]  B. Stengel,et al.  Team-Maxmin Equilibria☆ , 1997 .