Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft

An increasing number of our technological interactions are mediated through virtually embodied characters and software agents powered by machine learning. To understand how users relate to and evaluate these types of interfaces, we designed a Wizard of Oz prototype of an embodied agent in Minecraft that learns from users' actions, and conducted a user study with 18 school-aged Minecraft players. We categorised nine main ways users spontaneously attempted to interact with and teach the agent: four using game controls, and five using natural language text input. This study lays groundwork for a better understanding of human interaction with learning agents in virtual worlds.

[1]  Kristina Höök,et al.  Steps to take before intelligent user interfaces become real , 2000, Interact. Comput..

[2]  G. Stahl.  Theories of Cognition in Collaborative Learning , 2013 .

[3]  Saul Greenberg,et al.  Prototyping an intelligent agent through Wizard of Oz , 1993, INTERCHI.

[4]  Håkan Eftring,et al.  What Older People Expect of Robots: A Mixed Methods Approach , 2013, ICSR.

[5]  Kerstin Fischer,et al.  Effects of Different Kinds of Robot Feedback , 2013, ICSR.

[6]  Leila Takayama,et al.  Communication and knowledge sharing in human-robot interaction and learning from demonstration , 2010, Neural Networks.

[7]  Yong Xu,et al.  Active adaptation in human-agent collaborative interaction , 2011, Journal of Intelligent Information Systems.

[8]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[9]  Petra Wagner,et al.  Welcome to the Future - How Naïve Users Intuitively Address an Intelligent Robotics Apartment , 2016, ICSR.

[10]  Donald A. Norman,et al.  How might people interact with agents , 1994, CACM.

[11]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[12]  Manfred Tscheligi,et al.  Robots in Time: How User Experience in Human-Robot Interaction Changes over Time , 2013, ICSR.

[13]  Bradley C. Love,et al.  How Humans Teach Agents - A New Experimental Perspective , 2012, Int. J. Soc. Robotics.

[14]  Judith L. Green,et al.  Discourse Analysis, Learning, and Social Practice: A Methodological Study , 1998 .

[15]  Katja Hofmann,et al.  The Malmo Platform for Artificial Intelligence Experimentation , 2016, IJCAI.

[16]  Ben Shneiderman,et al.  The future of interactive systems and the emergence of direct manipulation , 1982 .

[17]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[18]  Clayton T. Morrison,et al.  Towards understanding how humans teach robots , 2011, UMAP'11.

[19]  Laurel D. Riek,et al.  Wizard of Oz studies in HRI , 2012, J. Hum. Robot Interact..

[20]  Pattie Maes,et al.  Agents that reduce work and information overload , 1994, CACM.