Human Learning in Atari

Atari games are an excellent testbed for studying intelligent behavior, as they offer a range of tasks that differ widely in their visual representation, game dynamics, and goals presented to an agent. The last two years have seen a spate of research into artificial agents that use a single algorithm to learn to play these games. The best of these artificial agents perform at better-than-human levels on most games, but require hundreds of hours of game-play experience to produce such behavior. Humans, on the other hand, can learn to perform well on these tasks in a matter of minutes. In this paper we present data on human learning trajectories for several Atari games, and test several hypotheses about the mechanisms that lead to such rapid learning.

[1]  L. Rips.  Inductive judgments about natural categories. , 1975 .

[2]  D. Medin,et al.  The role of theories in conceptual coherence. , 1985, Psychological review.

[3]  S. Carey.  Conceptual Change in Childhood , 1985 .

[4]  Linda B. Smith,et al.  The importance of shape in early lexical learning , 1988 .

[5]  Ellen M. Markman,et al.  Categorization and Naming in Children: Problems of Induction , 1989 .

[6]  Elizabeth S. Spelke,et al.  Principles of Object Perception , 1990, Cogn. Sci..

[7]  T. B. Ward.  Structured Imagination: the Role of Category Structure in Exemplar Generation , 1994, Cognitive Psychology.

[8]  B. Ross,et al.  Predictions From Uncertain Categorizations , 1994, Cognitive Psychology.

[9]  S. Carey,et al.  Whose gaze will infants follow? The elicitation of gaze-following in 12-month-olds , 1998 .

[10]  P. Bloom.  How children learn the meanings of words , 2000 .

[11]  Patrice D. Tremoulet,et al.  Perception of Animacy from the Motion of a Single Object , 2000, Perception.

[12]  György Gergely,et al.  One-year-old infants use teleological representations of actions productively , 2003, Cogn. Sci..

[13]  R. Baillargeon.  Infants' Physical World , 2004 .

[14]  A. Schlottmann,et al.  Perceived physical and social causality in animated motions: spontaneous reports and ratings. , 2006, Acta psychologica.

[15]  Katherine D. Kinzler,et al.  Core knowledge. , 2007, Developmental science.

[16]  J. Tenenbaum,et al.  Word learning as Bayesian inference. , 2007, Psychological review.

[17]  G. Csibra.  Goal attribution to inanimate agents by 6.5-month-old infants , 2008, Cognition.

[18]  R. Baillargeon,et al.  An Account of Infants' Physical Reasoning , 2008 .

[19]  T. Lombrozo.  Explanation and categorization: How “why?” informs “what?” , 2009, Cognition.

[20]  Joseph Jay Williams,et al.  The role of explanation in discovery and generalization: evidence from category learning , 2010, ICLS.

[21]  Charles Kemp,et al.  A probabilistic account of exemplar and category generation , 2013, Cognitive Psychology.

[22]  Noah D. Goodman,et al.  The mentalistic basis of core social cognition: experiments in preverbal infants and a computational model. , 2013, Developmental science.

[23]  Honglak Lee,et al.  Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning , 2014, NIPS.

[24]  Susan J. Hespos,et al.  Divisions of the physical world: Concepts of objects and substances. , 2015, Psychological bulletin.

[25]  Sergey Levine,et al.  Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models , 2015, ArXiv.

[26]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[27]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[28]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[29]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[30]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.