Mapping hearthstone deck spaces through MAP-elites with sliding boundaries

Quality diversity (QD) algorithms such as MAP-Elites have emerged as a powerful alternative to traditional single-objective optimization methods. They were initially applied to evolutionary robotics problems such as locomotion and maze navigation, but have yet to see widespread application. We argue that these algorithms are perfectly suited to the rich domain of video games, which contains many relevant problems with a multitude of successful strategies and often also multiple dimensions along which solutions can vary. This paper introduces a novel modification of the MAP-Elites algorithm called MAP-Elites with Sliding Boundaries (MESB) and applies it to the design and rebalancing of Hearthstone, a popular collectible card game chosen for its number of multidimensional behavior features relevant to particular styles of play. To avoid overpopulating cells with conflated behaviors, MESB slides the boundaries of cells based on the distribution of evolved individuals. Experiments in this paper demonstrate the performance of MESB in Hearthstone. Results suggest MESB finds diverse ways of playing the game well along the selected behavioral dimensions. Further analysis of the evolved strategies reveals common patterns that recur across behavioral dimensions and explores how MESB can help rebalance the game.

[1]  Keshav P. Dahal,et al.  Symbolic Reasoning for Hearthstone , 2018, IEEE Transactions on Games.

[2]  Lukasz Grad,et al.  Helping AI to play hearthstone using neural networks , 2017, 2017 Federated Conference on Computer Science and Information Systems (FedCSIS).

[3]  Aaron Isaksen.  Characterising Score Distributions in Dice Games , 2016 .

[4]  Julian Togelius,et al.  Generating beginner heuristics for simple texas hold'em , 2018, GECCO.

[5]  Zoran Popovic,et al.  Evaluating Competitive Game Balance with Restricted Play , 2012, AIIDE.

[6]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.

[7]  Joe Marks,et al.  Automatic Design of Balanced Board Games , 2007, AIIDE.

[8]  Dominik Slezak,et al.  Toward an Intelligent HS Deck Advisor: Lessons Learned from AAIA’ 18 Data Mining Competition , 2018, 2018 Federated Conference on Computer Science and Information Systems (FedCSIS).

[9]  Mark O. Riedl,et al.  Learning How Design Choices Impact Gameplay Behavior , 2019, IEEE Transactions on Games.

[10]  Julian Togelius,et al.  Evolving card sets towards balancing dominion , 2012, 2012 IEEE Congress on Evolutionary Computation.

[11]  Julian Togelius,et al.  Drawing without replacement as a game mechanic , 2018, FDG.

[12]  Günter Rudolph,et al.  Demonstrating the Feasibility of Automatic Game Balancing , 2016, GECCO.

[13]  Juan Julián Merelo Guervós,et al.  Automated playtesting in collectible card games using evolutionary algorithms: A case study in hearthstone , 2018, Knowl. Based Syst..

[14]  Kenneth O. Stanley,et al.  Evolving a diversity of virtual creatures through novelty search and local competition , 2011, GECCO '11.

[15]  Thomas Pfeiffer,et al.  Integrated Balancing of an RTS Game: Case Study and Toolbox Refinement , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[16]  Francisco S. Melo,et al.  Monte Carlo tree search experiments in hearthstone , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[17]  Kenneth O. Stanley,et al.  Quality Diversity: A New Frontier for Evolutionary Computation , 2016, Front. Robot. AI.

[18]  Kenneth O. Stanley,et al.  Exploiting Open-Endedness to Solve Problems Through the Search for Novelty , 2008, ALIFE.

[19]  Jakub Kowalski,et al.  Strategic Features and Terrain Generation for Balanced Heroes of Might and Magic III Maps , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[20]  Antoine Cully,et al.  Evolving a Behavioral Repertoire for a Walking Robot , 2013, Evolutionary Computation.

[21]  Daniele Gravina,et al.  Constrained surprise search for content generation , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[22]  Julian Togelius,et al.  Generating heuristics for novice players , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[23]  Julian Togelius,et al.  Generating Novice Heuristics for Post-Flop Poker , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[24]  Johannes Maucher,et al.  Hearthstone deck-construction with a utility system , 2016, 2016 10th International Conference on Software, Knowledge, Information Management & Applications (SKIMA).

[25]  Sverre Johann Bjørke,et al.  Deckbuilding in Magic: The Gathering Using a Genetic Algorithm , 2017 .

[26]  Joris Dormans,et al.  Simulating Mechanics to Study Emergence in Games , 2011, IDP@AIIDE.

[27]  Julian Togelius,et al.  Talakat: bullet hell generation through constrained map-elites , 2018, GECCO.

[28]  Mark O. Riedl,et al.  Monte-Carlo Tree Search for Simulation-based Strategy Analysis , 2019, ArXiv.

[29]  J. Togelius,et al.  Discovering Unique Game Variants , 2015 .

[30]  Frédéric Maire,et al.  Evolutionary Game Design , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[31]  Yuanzhe Jin.  Proposed Balance Model for Card Deck Measurement in Hearthstone , 2018, The Computer Games Journal.

[32]  Andrzej Janusz,et al.  Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithms , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[33]  Zhengxing Chen,et al.  Q-DeckRec: A Fast Deck Recommendation System for Collectible Card Games , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[34]  Andrzej Janusz,et al.  Helping AI to Play Hearthstone: AAIA'17 Data Mining Challenge , 2017, 2017 Federated Conference on Computer Science and Information Systems (FedCSIS).

[35]  Julian Togelius,et al.  An experiment in automatic game design , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[36]  Jean-Baptiste Mouret.  Novelty-Based Multiobjectivization , 2011 .

[37]  Alysson Ribeiro da Silva,et al.  HearthBot: An Autonomous Agent Based on Fuzzy ART Adaptive Neural Networks for the Digital Collectible Card Game HearthStone , 2018, IEEE Transactions on Games.

[38]  Michael Buro,et al.  Improving hearthstone AI by learning high-level rollout policies and bucketing chance node events , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[39]  Mark O. Riedl,et al.  Monte-Carlo Tree Search for Simulation-based Play Strategy Analysis , 2015, FDG.

[40]  Julian Togelius,et al.  Exploring the hearthstone deck space , 2018, FDG.

[41]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[42]  Elie Bursztein,et al.  I am a legend: Hacking hearthstone using statistical learning methods , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[43]  Dominik Slezak,et al.  Investigating Similarity between Hearthstone Cards: Text Embeddings and Interchangeability Approaches , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[44]  Juan Julián Merelo Guervós,et al.  Evolutionary deckbuilding in hearthstone , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[45]  Rudolf Kruse,et al.  Predicting Opponent Moves for Improving Hearthstone AI , 2018, IPMU.

[46]  Jan Jakubik.  Evaluation of hearthstone game states with neural networks and sparse autoencoding , 2017, 2017 Federated Conference on Computer Science and Information Systems (FedCSIS).

[47]  Antoine Cully,et al.  Behavioral repertoire learning in robotics , 2013, GECCO '13.