Crowdsourcing for closed-loop control

We present a system for large scale robotic learning from demonstration. We describe a set of software tools for enabling human-robot interaction over the internet and gathering the large datasets that such crowdsourcing makes possible. We show results in which humans teach a robot to navigate a maze over the Internet. Robots occupy a peculiar place in our culture. We have been building robots in our imaginations for decades, robots that are alternately wondrous or terrifying, always brilliant and consummately skilled. In contrast, real robots are typically brittle and capable of only a few simple constrained tasks. Additionally, only expert programmers, intimately familiar with the particulars of a low-level robotic system, can hope to achieve any kind of complex robot behavior. This paper describes our efforts to apply the lessons of crowdsourcing to robotics to leverage the power and knowledge of a truly large number of end users to create more skilled and robust robot controllers. We focus on learning from demonstration (LfD), [2, 1] an approach to robot programming in which users demonstrate desired skills to a robot. Nothing is required of the user beyond the ability to complete the task in a way that the robot can interpret. Traditionally, LfD research has been constrained by the number of demonstrations that can be performed; unless a large number of users can interact with the robot only a limited amount of data can easily be gathered. Since users do not usually need specialized skills to demonstrate robot skills, a web-enabled system could be used to collect data from a large number of users. An online system also lifts the burden of training the robot from a single user who may only want to contribute a few demonstrations. Collecting data for closed-loop control is different from many crowdsourcing applications examined in the Machine Learning community, which has typically focused on annotation of text and labeling of images. Task demonstration often requires a significant interaction both in terms of time and information provided to the user. Users not only give the robot instructions, but also evaluate the results and provide new instructions given the outcome. We describe a recently developed system that allows a large number of users to train a robot to solve a task (in this case maze-navigation) through a video-game style interface. While a single demonstration may contain errors and provides only limited data, the demonstrations from multiple users provide enough data to create a robust policy. There have been a few initial efforts in putting robots on the Internet [4, 9]. These approaches generally allowed people to interact with robots but were not aimed at task learning. Other work examined using crowdsourcing approaches to train robots through game playing environments. Chernova et al examined using a multi-player video game where users collaborate to provide user demonstrations [3]. This work differs from ours in that