Learning to Play: Reinforcement Learning and Games

principled frameworks such as minimax, reinforcement learning, or function approximation. In addition to the elegant conceptual frameworks, deep, dirty, domain-specific understanding is necessary for progress in this field [594]. 8.1.4 Towards General Intelligence Let us revisit the problem statement from the Introduction. What are the machine learning methods that are used in Chess and Go to achieve a level of play stronger than the strongest humans? Various reinforcement learning methods have beaten human champions in Chess and Go, ranging from heuristic planning, adaptive sampling, function approximation, to self-play. All methods fit in the classic search-eval architecture, and all use variants of the principles of bootstrapping and optimization. Heuristic planning can achieve levels of play far surpassing human ability in Chess and Checkers. We have also seen how to combine planning and learning into a self-play system that learns to play from scratch, achieving even higher levels in Chess, Shogi, and Go. Did we achieve intelligence? For these three games, we certainly passed the Turing test.1 Systems were created that behave at a level for which a human would need super human intelligence. The intelligence is, however, single-domain intelligence. AlphaGo cannot play Shogi, and it cannot think of an entertaining joke. Joining the two fields of symbolic AI and connectionist AI, AlphaZero showed that the methods generalize to three games, hinting at artificial general intelligence. However, it is more precise to speak of training three almost identical systems to become three different systems specialized in their own kind of special intelligence (since the net, once trained for Go, cannot play Chess). (GGP systems can play general games, but do not achieve performance close to AlphaZero; Sect. 8.2.5.) 8.2 Future Drosophilas The reinforcement learning results in Atari andGo have inspiredmuch further research. The use of concrete games, the Drosophilas, has stimulated progress in game playing greatly. In addition to the usual two-person zero-sum perfect-information games, researchers have looked for games that capture other elements of real life, such as non-zero-sum (collaboration) games, imperfect information, and multi-agent games. In this section we will review some of these aspects. 1 Actually, since we far surpassed human play, an argument can be made that in doing so we failed the Turing test, since no human can play at this level.