A novel real-time design for fighting game AI

Real-time fighting games are challenging for computer agents in that actions must be decided within a relatively short cycle of time, usually in milliseconds or less. That is only achievable by either very powerful machines or state-of-the-art algorithms. The former is usually a costly option while the latter remains an ongoing research topic despite countless research. This paper describes our algorithmic approach towards real-time fighting games via the fighting game AI challenge. The focus of our research is the LUD division, the most challenging category of the competition where action data is hidden to prevent methods that are dependent on prior training. In this paper, we propose several generic heuristics that can be used in combination with Monte-Carlo tree search. Our experimental results show that such an approach would provide an excellent AI outperforming pure Monte-Carlo tree search and classic algorithms such as evolutionary algorithms or deep reinforcement learning. Nonetheless, we believe that our proposed heuristics should be able to generalize to other domains beyond the scope of the fighting game AI challenge.

[1]  M Tamassia,et al.  Artificial intelligence techniques towards adaptive digital games , 2017 .

[2]  Ruck Thawonmas,et al.  Monte-Carlo Tree Search Implementation of Fighting Game AIs Having Personas , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[3]  Doina Logofatu,et al.  General Video Game AI Controller - Integrating Three Algorithms to Bring a New Solution , 2019, 2019 23rd International Conference on System Theory, Control and Computing (ICSTCC).

[4]  Chang Wook Ahn,et al.  Hybrid fighting game AI using a genetic algorithm and Monte Carlo tree search , 2018, GECCO.

[5]  Jasper Marianczuk,et al.  Engineering faster sorters for small sets of items , 2019, Softw. Pract. Exp..

[6]  Julian Togelius,et al.  General Video Game AI: Competition, Challenges and Opportunities , 2016, AAAI.

[7]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[8]  Luciano Reis Coutinho,et al.  Hierarchical Reinforcement Learning With Monte Carlo Tree Search in Computer Fighting Game , 2019, IEEE Transactions on Games.

[9]  Ruck Thawonmas,et al.  Applying and Improving Monte-Carlo Tree Search in a Fighting Game AI , 2016, ACE.

[10]  Kyung-Joong Kim,et al.  Deep Q networks for visual fighting game AI , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[11]  Doina Logofatu,et al.  Machine Learning with the Pong Game: A Case Study , 2018, EANN.

[12]  Sung Hoon Jung,et al.  Exploiting Intelligence in Fighting Action Games Using Neural Networks , 2006, IEICE Trans. Inf. Syst..

[13]  Ruck Thawonmas,et al.  Investigation of Various Online Adaptation Methods of Computer-Game AI Rulebase in Dynamic Scripting , 2006 .

[14]  Xiaodong Li,et al.  Monte Carlo tree search based algorithms for dynamic difficulty adjustment , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[15]  Doina Logofatu,et al.  Developing a General Video Game AI Controller Based on an Evolutionary Approach , 2019, ACIIDS.

[16]  Chang Wook Ahn,et al.  Integrating agent actions with genetic action sequence method , 2019, GECCO.

[17]  Ruck Thawonmas,et al.  A method for online adaptation of computer-game AI rulebase , 2006, ACE '06.

[18]  Benjamin Wai-ming Ng,et al.  Street Fighter and The King of Fighters in Hong Kong: A Study of Cultural Consumption and Localization of Japanese Games in an Asian Context , 2006, Game Studies.

[19]  Rosa Maria Valdovinos,et al.  Some Variations of Upper Confidence Bound for General Game Playing , 2019, MCPR.

[20]  Ioannis P. Vlahavas,et al.  Reinforcement learning agents providing advice in complex video games , 2014, Connect. Sci..

[21]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[22]  Giovanna Martínez-Arellano,et al.  Creating AI Characters for Fighting Games Using Genetic Programming , 2017, IEEE Transactions on Computational Intelligence and AI in Games.

[23]  Kyung-Joong Kim,et al.  Opponent modeling based on action table for MCTS-based fighting game AI , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[24]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[25]  Ruck Thawonmas,et al.  Online adjustment of the AI's strength in a fighting game using the k-nearest neighbor algorithm and a game simulator , 2014, 2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE).

[26]  Pieter Spronck,et al.  Monte-Carlo Tree Search: A New Framework for Game AI , 2008, AIIDE.

[27]  Costin Badica,et al.  A Hybrid Approach for the Fighting Game AI Challenge: Balancing Case Analysis and Monte Carlo Tree Search for the Ultimate Performance in Unknown Environment , 2019, EANN.

[28]  Mark O. Riedl,et al.  Monte-Carlo Tree Search for Simulation-based Strategy Analysis , 2019, ArXiv.

[29]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[30]  Ruck Thawonmas,et al.  Predicting the opponent's action using the k-nearest neighbor algorithm and a substring tree structure , 2015, 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE).

[31]  Spyridon Samothrakis,et al.  On Monte Carlo Tree Search and Reinforcement Learning , 2017, J. Artif. Intell. Res..

[32]  Marc Cavazza,et al.  Al in computer games: Survey and perspectives , 2000, Virtual Reality.

[33]  Ruck Thawonmas,et al.  Application of Monte-Carlo tree search in a fighting game AI , 2016, 2016 IEEE 5th Global Conference on Consumer Electronics.

[34]  Simon M. Lucas,et al.  Ms. Pac-Man Versus Ghost Team CIG 2016 competition , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[35]  H. Jaap van den Herik,et al.  Cross-Entropy for Monte-Carlo Tree Search , 2008, J. Int. Comput. Games Assoc..

[36]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[37]  Luiz Chaimowicz,et al.  Discovering Combos in Fighting Games with Evolutionary Algorithms , 2016, GECCO.

[38]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[39]  Ruck Thawonmas,et al.  Fighting game artificial intelligence competition platform , 2013, 2013 IEEE 2nd Global Conference on Consumer Electronics (GCCE).

[40]  Geoff Skinner,et al.  Artificial Intelligence and Deep Learning in Video Games A Brief Review , 2019, 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS).

[41]  H. Jaap van den Herik,et al.  Parallel Monte-Carlo Tree Search , 2008, Computers and Games.

[42]  Ruck Thawonmas,et al.  Monte-Carlo Tree Search for Implementation of Dynamic Difficulty Adjustment Fighting Game AIs Having Believable Behaviors , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).