Measuring Intelligence through Games

Abstract Artificial general intelligence (AGI) refers to research aimed at tackling the full problemof artificial intelligence, that is, create truly intelligent agents. This sets it apart from mostAI research which aims at solving relatively narrow domains, such as character recognition,motion planning, or increasing player satisfaction in games. But how do we know when anagent is truly intelligent? A common point of reference in the AGI community is Legg andHutter’s formal definition of universal intelligence, which has the appeal of simplicity andgenerality but is unfortunately incomputable.Games of various kinds are commonly used as benchmarks for “narrow” AI research,as they are considered to have many important properties. We argue that many of theseproperties carry over to the testing of general intelligence as well. We then sketch how suchtesting could practically be carried out. The central part of this sketch is an extension ofuniversal intelligence to deal with finite time, and the use of sampling of the space of gamesexpressed in a suitably biased game description language.Keywords: measure of intelligence, games

[1]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[2]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[3]  Hilla Peretz,et al.  Ju n 20 03 Schrödinger ’ s Cat : The rules of engagement , 2003 .

[4]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[5]  Gerard Casey.  Minds and machines , 1992 .

[6]  John E. Laird,et al.  Human-Level AI's Killer Application: Interactive Computer Games , 2000, AI Mag..

[7]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .

[8]  Steffen Christensen,et al.  The Turing Ratio: Metrics For Open-ended Tasks , 2002, GECCO.

[9]  H. Masum,et al.  The Turing Ratio : A Framework for Open-Ended Task Metrics , 2003 .

[10]  Jürgen Schmidhuber,et al.  Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , 2003, ArXiv.

[11]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[12]  Katie Salen,et al.  Rules of play: game design fundamentals , 2003 .

[13]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[14]  Marcus Hutter.  Simulation Algorithms for Computational Systems Biology , 2017, Texts in Theoretical Computer Science. An EATCS Series.

[15]  Richard E. Korf,et al.  Frontier search , 2005, JACM.

[16]  Jürgen Schmidhuber.  2006: Celebrating 75 Years of AI - History and Outlook: The Next 25 Years , 2006, 50 Years of Artificial Intelligence.

[17]  Shane Legg,et al.  Universal Intelligence: A Definition of Machine Intelligence , 2007, Minds and Machines.

[18]  S. Legg,et al.  Machine super intelligence , 2008 .

[19]  Julian Togelius,et al.  An experiment in automatic game design , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[20]  Julian Togelius,et al.  The WCCI 2008 simulated car racing competition , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[21]  Julian Togelius,et al.  The 2007 IEEE CEC simulated car racing competition , 2008, Genetic Programming and Evolvable Machines.

[22]  Julian Togelius,et al.  Mario AI competition , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[23]  Philip Hingston,et al.  A Turing Test for Computer Game Bots , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[24]  Jürgen Schmidhuber,et al.  Ultimate Cognition à la Gödel , 2009, Cognitive Computation.

[25]  J. Schmidhuber,et al.  Frontier Search , 2009 .

[26]  Julian Togelius,et al.  The 2009 Mario AI Competition , 2010, IEEE Congress on Evolutionary Computation.

[27]  José Hernández-Orallo,et al.  On Evaluating Agent Performance in a Fixed Period of Time (Extended Version) , 2010, AGI 2010.

[28]  Ben Goertzel.  Toward a Formal Characterization of Real-World General Intelligence , 2010, AGI 2010.

[29]  Frédéric Maire,et al.  Evolutionary Game Design , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[30]  Julian Togelius,et al.  The 2009 Simulated Car Racing Championship , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[31]  Joel Veness,et al.  A Monte-Carlo AIXI Approximation , 2009, J. Artif. Intell. Res..