Investigating Human Priors for Playing Video Games

What makes humans so good at solving seemingly complex video games? Unlike computers, humans bring in a great deal of prior knowledge about the world, enabling efficient decision making. This paper investigates the role of human priors for solving video games. Given a sample game, we conduct a series of ablation studies to quantify the importance of various priors on human performance. We do this by modifying the video game environment to systematically mask different types of visual information that could be used by humans as priors. We find that removal of some prior knowledge causes a drastic degradation in the speed with which human players solve the game, e.g. from 2 minutes to over 20 minutes. Furthermore, our results indicate that general priors, such as the importance of objects and visual consistency, are critical for efficient game-play. Videos and the game manipulations are available at this https URL

[1]  Elizabeth S. Spelke,et al.  Principles of Object Perception , 1990, Cogn. Sci..

[2]  R. Baillargeon.  How Do Infants Learn About the Physical World? , 1994 .

[3]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[4]  P. Quinn,et al.  Categorization in infancy , 2001, Trends in Cognitive Sciences.

[5]  R. Baillargeon.  Infants' Physical World , 2004 .

[6]  S. Carey.  The Origin of Concepts , 2000 .

[7]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[8]  Katherine D. Kinzler,et al.  Core knowledge. , 2007, Developmental science.

[9]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[10]  Andre Cohen,et al.  An object-oriented representation for efficient reinforcement learning , 2008, ICML '08.

[11]  Linda B. Smith,et al.  Developmental changes in visual object recognition between 18 and 24 months of age. , 2009, Developmental science.

[12]  Susan J. Hespos,et al.  PSYCHOLOGICAL SCIENCE Research Article Five-Month-Old Infants Have Different Expectations for Solids and Liquids , 2022 .

[13]  Noah D. Goodman,et al.  The double-edged sword of pedagogy: Instruction limits spontaneous exploration and discovery , 2011, Cognition.

[14]  Zoubin Ghahramani,et al.  A Comparison of Human and Agent Reinforcement Learning in Partially Observable Domains , 2011, CogSci.

[15]  P. Stone,et al.  The Nature of Belief-Directed Exploratory Choice in Human Decision-Making , 2011, Front. Psychology.

[16]  Kari S. Kretch,et al.  Cliff or step? Posture-specific learning at the edge of a drop-off. , 2013, Child development.

[17]  James J. Gibson,et al.  The Ecological Approach to Visual Perception: Classic Edition , 2014 .

[18]  Christopher G. Lucas,et al.  When children are better (or at least more open-minded) learners than adults: Developmental differences in learning the forms of causal relationships , 2014, Cognition.

[19]  Samuel Gershman,et al.  Novelty and Inductive Generalization in Human Reinforcement Learning , 2015, Top. Cogn. Sci..

[20]  Peter Kulchyski.  and , 2015 .

[21]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[22]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[23]  Tom Schaul,et al.  Unifying Count-Based Exploration and Intrinsic Motivation , 2016, NIPS.

[24]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[25]  Sergey Levine,et al.  Continuous Deep Q-Learning with Model-based Acceleration , 2016, ICML.

[26]  Alexei A. Efros,et al.  Curiosity-Driven Exploration by Self-Supervised Prediction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Joshua B. Tenenbaum,et al.  Human Learning in Atari , 2017, AAAI Spring Symposia.

[28]  Dileep George,et al.  Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics , 2017, ICML.

[29]  Satinder Singh,et al.  Value Prediction Network , 2017, NIPS.

[30]  Regina Barzilay,et al.  Deep Transfer in Reinforcement Learning by Language Grounding , 2017, ArXiv.

[31]  Regina Barzilay,et al.  Grounding Language for Transfer in Deep Reinforcement Learning , 2017, J. Artif. Intell. Res..