Survey of Artificial Intelligence for Card Games and Its Application to the Swiss Game Jass

In the last decades we have witnessed the success of applications of Artificial Intelligence to playing games. In this work we address the challenging field of games with hidden information and card games in particular. Jass is a very popular card game in Switzerland and is closely connected with Swiss culture. To the best of our knowledge, performances of Artificial Intelligence agents in the game of Jass do not outperform top players yet. Our contribution to the community is two-fold. First, we provide an overview of the current state-of-the-art of Artificial Intelligence methods for card games in general. Second, we discuss their application to the use-case of the Swiss card game Jass. This paper aims to be an entry point for both seasoned researchers and new practitioners who want to join in the Jass challenge.

[1]  Jason Noble,et al.  Finding Robust Texas Hold'em Poker Strategies Using Pareto Coevolution and Deterministic Crowding , 2002, ICMLA.

[2]  Malte Helmert,et al.  A Skat Player Based on Monte-Carlo Simulation , 2006, Computers and Games.

[3]  Andrzej Janusz,et al.  Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithms , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[4]  Nathan R. Sturtevant,et al.  An Analysis of UCT in Multi-Player Games , 2008, J. Int. Comput. Games Assoc..

[5]  David Silver,et al.  Smooth UCT Search in Computer Poker , 2015, IJCAI.

[6]  Tuomas Sandholm,et al.  Deep Counterfactual Regret Minimization , 2018, ICML.

[7]  Yoshimasa Tsuruoka,et al.  Neural Fictitious Self-Play in Imperfect Information Games with Many Players , 2017, CGW@IJCAI.

[8]  Ian D. Watson,et al.  Computer poker: A review , 2011, Artif. Intell..

[9]  Hirotaka Osawa,et al.  Solving Hanabi: Estimating Hands by Opponent's Actions in Cooperative Game with Incomplete Information , 2015, AAAI Workshop: Computer Poker and Imperfect Information.

[10]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[11]  Peter I. Cowling,et al.  Integrating Monte Carlo Tree Search with Knowledge-Based Methods to Create Engaging Play in a Commercial Mobile Game , 2013, AIIDE.

[12]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[13]  Peter I. Cowling,et al.  Information Set Monte Carlo Tree Search , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[14]  Michael H. Bowling,et al.  Using Response Functions to Measure Strategy Strength , 2014, AAAI.

[15]  Kevin Waugh,et al.  Monte Carlo Sampling for Regret Minimization in Extensive Games , 2009, NIPS.

[16]  Branislav Bosanský,et al.  Practical Performance of Refinements of Nash Equilibria in Extensive-Form Zero-Sum Games , 2014, ECAI.

[17]  Tuomas Sandholm,et al.  Solving Imperfect-Information Games via Discounted Regret Minimization , 2018, AAAI.

[18]  Stefano Di Palma,et al.  Traditional Wisdom and Monte Carlo Tree Search Face-to-Face in the Card Game Scopone , 2018, IEEE Transactions on Games.

[19]  Yoshimasa Tsuruoka,et al.  Building a computer Mahjong player based on Monte Carlo simulation and opponent models , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[20]  David Silver,et al.  Deep Reinforcement Learning from Self-Play in Imperfect-Information Games , 2016, ArXiv.

[21]  Michael Buro,et al.  Improving Search with Supervised Learning in Trick-Based Card Games , 2019, AAAI.

[22]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[23]  Peter I. Cowling,et al.  Ensemble Determinization in Monte Carlo Tree Search for the Imperfect Information Card Game Magic: The Gathering , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[24]  Peter I. Cowling,et al.  Monte Carlo search applied to card selection in Magic: The Gathering , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[25]  Noam Brown,et al.  Superhuman AI for heads-up no-limit poker: Libratus beats top professionals , 2018, Science.

[26]  Malte Helmert,et al.  A Doppelkopf Player Based on UCT , 2015, KI.

[27]  Marc Lanctot,et al.  Computing Approximate Nash Equilibria and Robust Best-Responses Using Sampling , 2011, J. Artif. Intell. Res..

[28]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[29]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[30]  Matthew L. Ginsberg,et al.  GIB: Imperfect Information in a Computationally Challenging Game , 2011, J. Artif. Intell. Res..

[31]  Sam Devlin,et al.  Player Preference and Style in a Leading Mobile Card Game , 2015, IEEE Transactions on Computational Intelligence and AI in Games.

[32]  Neil Burch,et al.  Heads-up limit hold’em poker is solved , 2015, Science.

[33]  Sam Devlin,et al.  Combining Gameplay Data with Monte Carlo Tree Search to Emulate Human Play , 2016, AIIDE.

[34]  S. Edelkamp.  Challenging Human Supremacy in Skat – Guided and Complete And-Or Belief-Space Tree Search for Solving the Nullspiel , 2018 .

[35]  Francisco S. Melo,et al.  Monte Carlo tree search experiments in hearthstone , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[36]  Tuomas Sandholm,et al.  Solving Large Sequential Games with the Excessive Gap Technique , 2018, NeurIPS.

[37]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[38]  Tsan-sheng Hsu,et al.  Convergence and correctness analysis of Monte-Carlo tree search algorithms: A case study of 2 by 4 Chinese dark chess , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[39]  Simon M. Lucas,et al.  Evaluating and modelling Hanabi-playing agents , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[40]  Cyril Fonlupt,et al.  Monte-Carlo Tree Search for the Game of "7 Wonders" , 2014, CGW@ECAI.

[41]  Walter A. Kosters,et al.  Aspects of the Cooperative Card Game Hanabi , 2016, BNCAI.

[42]  Julian Togelius,et al.  Evolving card sets towards balancing dominion , 2012, 2012 IEEE Congress on Evolutionary Computation.

[43]  Neil Burch,et al.  Time and Space: Why Imperfect Information Games are Hard , 2018 .

[44]  Peter I. Cowling,et al.  Determinization and information set Monte Carlo Tree Search for the card game Dou Di Zhu , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[45]  J. Schaeffer,et al.  Comparing UCT versus CFR in Simultaneous Games , 2009 .

[46]  Henry Charlesworth,et al.  Application of Self-Play Reinforcement Learning to a Four-Player Game of Imperfect Information , 2018, ArXiv.

[47]  Alan Fern,et al.  Lower Bounding Klondike Solitaire with Monte-Carlo Planning , 2009, ICAPS.

[48]  Kevin Waugh,et al.  DeepStack: Expert-level artificial intelligence in heads-up no-limit poker , 2017, Science.

[49]  Benjamin Van Roy,et al.  Solitaire: Man Versus Machine , 2004, NIPS.

[50]  Michael H. Bowling,et al.  Online Monte Carlo Counterfactual Regret Minimization for Search in Imperfect Information Games , 2015, AAMAS.

[51]  Nathan R. Sturtevant,et al.  Feature Construction for Reinforcement Learning in Hearts , 2006, Computers and Games.