The aggregate complexity of decisions in the game of Go

Abstract. Artificial intelligence (AI) research is fast approaching, or perhaps has already reached, a bottleneck whereby further advancement towards practical human-like reasoning in complex tasks needs further quantified input from large studies of human decision-making. Previous studies in psychology, for example, often rely on relatively small cohorts and very specific tasks. These studies have strongly influenced some of the core notions in AI research such as the reinforcement learning and the exploration versus exploitation paradigms. With the goal of contributing to this direction in AI developments we present our findings on the evolution towards world-class decision-making across large cohorts of subjects in the formidable game of Go. Some of these findings directly support previous work on how experts develop their skills but we also report on several previously unknown aspects of the development of expertise that suggests new avenues for AI research to explore. In particular, at the level of play that has so far eluded current AI systems for Go, we are able to quantify the lack of ‘predictability’ of experts and how this changes with their level of skill.

[1]  J. Pine,et al.  Chunking mechanisms in human learning , 2001, Trends in Cognitive Sciences.

[2]  Hua Wang,et al.  Data and Knowledge Engineering , 2012, Lecture Notes in Computer Science.

[3]  David Silver,et al.  Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008) Achieving Master Level Play in 9 × 9 Computer Go , 2022 .

[4]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[5]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[6]  Sheng He,et al.  A functional MRI study of high-level cognition. I. The game of chess. , 2003, Brain research. Cognitive brain research.

[7]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[8]  Feng-hsiung Hsu,et al.  Search Control Methods in Deep Blue , 2002 .

[9]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[10]  Yuhong Yang,et al.  Information Theory, Inference, and Learning Algorithms , 2005 .

[11]  Rainer Bromme,et al.  Professional learning: Gaps and transitions on the way from novice to expert , 2004 .

[12]  Judith S Reitman,et al.  Skilled perception in Go: Deducing memory structures from inter-response times , 1976, Cognitive Psychology.

[13]  Junichiro Yoshimoto,et al.  Control of exploitation-exploration meta-parameter in reinforcement learning , 2002, Neural Networks.

[14]  N. Charness,et al.  Visual Span in Expert Chess Players: Evidence From Eye Movements , 2001, Psychological science.

[15]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[16]  Jorma Rissanen,et al.  Minimum Description Length Principle , 2010, Encyclopedia of Machine Learning.

[17]  H. Simon,et al.  Pattern recognition makes search possible: Comments on Holding (1992) , 1998 .

[18]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[19]  H. Schmidt,et al.  Knowledge restructuring in expertise development: Evidence from pathophysiological representations of clinical cases by students and physicians , 2000 .

[20]  A. Vulpiani,et al.  Predictability: a way to characterize complexity , 2001, nlin/0101029.

[21]  Dennis H. Holding,et al.  Theories of chess skill , 1992 .

[22]  Kokolo Ikeda,et al.  Advances in Computer Games , 2011, Lecture Notes in Computer Science.

[23]  Marta Indulska,et al.  How do practitioners use conceptual modeling in practice? , 2006, Data Knowl. Eng..

[24]  James Ward,et al.  The British Journal of Psychology , 1904 .

[25]  Thomas M. Cover,et al.  Some equivalences between Shannon entropy and Kolmogorov complexity , 1978, IEEE Trans. Inf. Theory.

[26]  H. Simon,et al.  A simulation of memory for chess positions. , 1973 .

[27]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[28]  Donald C. Wunsch,et al.  Computer Go: A Grand Challenge to AI , 2007, Challenges for Computational Intelligence.

[29]  N. Charness.  Search in chess: Age and skill differences. , 1981 .

[30]  Janet Wiles,et al.  The challenge of Go as a domain for AI research: a comparison between Go and chess , 1995, Proceedings of Third Australian and New Zealand Conference on Intelligent Information Systems. ANZIIS-95.

[31]  Robert Miller,et al.  Learning and Individual Differences , 2010 .

[32]  D. Holding.  The Psychology of Chess Skill , 1985 .

[33]  Sylvain Gelly,et al.  Modifications of UCT and sequence-like simulations for Monte-Carlo Go , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[34]  Alfred A. Cleveland.  The Psychology of Chess and of Learning to Play It , 1907 .

[35]  M. Roulston.  Estimating the errors on measured entropy and mutual information , 1999 .

[36]  Susan L. Epstein,et al.  Learning Game-Specific Spatially-Oriented Heuristics , 1998, Constraints.

[37]  N. Charness.  The impact of chess research on cognitive science , 1992 .

[38]  Sheng He,et al.  A functional MRI study of high-level cognition , 2003 .

[39]  Thore Graepel,et al.  Learning on Graphs in the Game of Go , 2001, ICANN.

[40]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[41]  Elwyn R. Berlekamp,et al.  Mathematical Go - chilling gets the last point , 1994 .

[42]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[43]  H. Simon,et al.  Recall of rapidly presented random chess positions is a function of skill , 1996, Psychonomic bulletin & review.

[44]  William Bart,et al.  Moves in Mind: The Psychology of Board Games , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[45]  Miao‐kun Sun,et al.  Trends in cognitive sciences , 2012 .

[46]  O. Bagasra,et al.  Proceedings of the National Academy of Sciences , 1914, Science.

[47]  Drew H. Abney,et al.  Journal of Experimental Psychology : Human Perception and Performance Influence of Musical Groove on Postural Sway , 2015 .

[48]  Albert L. Zobrist,et al.  A model of visual organization for the game of GO , 1899, AFIPS '69 (Spring).

[49]  Tristan Cazenave,et al.  Combining Tactical Search and Monte-Carlo in the Game of Go , 2005, CIG.

[50]  Thore Graepel,et al.  Bayesian pattern ranking for move prediction in the game of Go , 2006, ICML.

[51]  John D. Van Horn,et al.  Characterizing mature human intelligence: Expertise development , 2000 .

[52]  Michael X. Cohen,et al.  Behavioral / Systems / Cognitive Reinforcement Learning Signals Predict Future Decisions , 2007 .

[53]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[54]  Kenji Doya,et al.  Meta-learning in Reinforcement Learning , 2003, Neural Networks.

[55]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[56]  H. Simon,et al.  Recall of random and distorted chess positions: Implications for the theory of expertise , 1996, Memory & cognition.

[57]  F. Gobet,et al.  Visuospatial abilities of chess players. , 2002, British journal of psychology.

[58]  S. Sanner.  Learning CRFs with Hierarchical Features : An Application to Go , 2007 .

[59]  H. Simon,et al.  Perception in chess , 1973 .

[60]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[61]  B. Bouzy.  Spatial Reasoning in the game of Go , 1996 .