Traditional Wisdom and Monte Carlo Tree Search Face-to-Face in the Card Game Scopone

We present the design of a competitive artificial intelligence for Scopone, a popular Italian card game. We compare rule-based players using the most established strategies (one for beginners and two for advanced players) against players using Monte Carlo Tree Search (MCTS) and Information Set Monte Carlo Tree Search (ISMCTS) with different reward functions and simulation strategies. MCTS requires complete information about the game state and thus implements a cheating player, whereas ISMCTS can deal with incomplete information and thus implements a fair player. Our results show that, as expected, the cheating MCTS outperforms all the other strategies; ISMCTS is stronger than all the rule-based players implementing well-known and most advanced strategies and it also turns out to be a challenging opponent for human players.

[1]  Peter I. Cowling,et al.  Determinization and information set Monte Carlo Tree Search for the card game Dou Di Zhu , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[2]  Julian Togelius,et al.  Modifying MCTS for Human-Like General Video Game Playing , 2016, IJCAI.

[3]  Julian Togelius,et al.  Artificial and Computational Intelligence in Games , 2013, Artificial and Computational Intelligence in Games.

[4]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[5]  Sam Devlin,et al.  Player Preference and Style in a Leading Mobile Card Game , 2015, IEEE Transactions on Computational Intelligence and AI in Games.

[6]  Guillaume Chaslot,et al.  A Comparison of Monte-Carlo Methods for Phantom Go , 2007 .

[7]  Dana S. Nau,et al.  Computer Bridge - A Big Win for AI Planning , 1998, AI Mag..

[8]  Peter I. Cowling,et al.  Integrating Monte Carlo Tree Search with Knowledge-Based Methods to Create Engaging Play in a Commercial Mobile Game , 2013, AIIDE.

[9]  Paolo Ciancarini,et al.  Monte Carlo tree search in Kriegspiel , 2010, Artif. Intell..

[10]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[11]  Mark J. Nelson,et al.  Investigating vanilla MCTS scaling on the GVG-AI game corpus , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[12]  Pieter Spronck,et al.  Monte-Carlo Tree Search in Settlers of Catan , 2009, ACG.

[13]  Ian D. Watson,et al.  Computer poker: A review , 2011, Artif. Intell..

[14]  Peter I. Cowling,et al.  Memory Bounded Monte Carlo Tree Search , 2017, AIIDE.

[15]  Paul M Bethe.  The State of Automated Bridge Play , 2009 .

[16]  Stefano Di Palma.  Monte Carlo tree search algorithms applied to the card game Scopone , 2014 .

[17]  Marcel Roeloffzen,et al.  Hanabi is NP-complete, Even for Cheaters who Look at Their Cards , 2016, FUN.

[18]  Daniele Loiacono,et al.  Player Modeling , 2013, Artificial and Computational Intelligence in Games.

[19]  Simon M. Lucas,et al.  Evaluating and modelling Hanabi-playing agents , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[20]  Cyril Fonlupt,et al.  Monte-Carlo Tree Search for the Game of "7 Wonders" , 2014, CGW@ECAI.

[21]  Computer Poker and Imperfect Information, Papers from the 2015 AAAI Workshop, Austin, Texas, USA, January 26, 2015 , 2015, AAAI Workshop: Computer Poker and Imperfect Information.

[22]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[23]  Julian Togelius,et al.  Monte-Carlo tree search for persona based player modeling , 2015 .

[24]  Guillaume Chaslot,et al.  Integrating Opponent Models with Monte-Carlo Tree Search in Poker , 2010, Interactive Decision Theory and Game Theory.

[25]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[26]  Tuomas Sandholm,et al.  Safe and Nested Subgame Solving for Imperfect-Information Games , 2017, NIPS.

[27]  Peter I. Cowling,et al.  Information Set Monte Carlo Tree Search , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[28]  Peter I. Cowling,et al.  Emergent bluffing and inference with Monte Carlo Tree Search , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[29]  Jonathan Schaeffer,et al.  The challenge of poker , 2002, Artif. Intell..

[30]  Julian Togelius,et al.  Investigating MCTS modifications in general video game playing , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[31]  Computer Poker and Imperfect Information Games, Papers from the 2016 AAAI Workshop, Phoenix, Arizona, USA, February 13, 2016 , 2016, AAAI Workshop: Computer Poker and Imperfect Information Games.

[32]  Hongye Li,et al.  Belief-state Monte-Carlo tree search for Phantom games , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[33]  Ryan B. Hayward,et al.  Monte Carlo Tree Search in Hex , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[34]  Tristan Cazenave,et al.  A Phantom-Go Program , 2006, ACG.

[35]  Mark H. M. Winands,et al.  Monte-Carlo Tree Search for the game of Scotland Yard , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[36]  Yoshimasa Tsuruoka,et al.  Building a computer Mahjong player based on Monte Carlo simulation and opponent models , 2015, 2015 IEEE Conference on Computational Intelligence and Games (CIG).

[37]  Peter I. Cowling,et al.  Ensemble Determinization in Monte Carlo Tree Search for the Imperfect Information Card Game Magic: The Gathering , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[38]  Peter I. Cowling,et al.  Heuristic move pruning in Monte Carlo Tree Search for the strategic card game Lords of War , 2014, 2014 IEEE Conference on Computational Intelligence and Games.