As more robots act in physical proximity to people, it is essential to ensure they make decisions and execute actions that align with human values. To do so, robots need to understand the true intentions behind human-issued commands. In this paper, we define a safe robot as one that receives a natural-language command from humans, considers an action in response to that command, and accurately predicts how humans will judge that action if is executed in reality. Our contribution is two-fold: First, we introduce a web platform for human users to propose commands to simulated robots. The robots receive commands and act based on those proposed commands, and then the users provide positive and/or negative reinforcement. Next, we train a critic for each robot to predict the crowd's responses to one of the crowd-proposed commands. Second, we show that the morphology of a robot plays a role in the way it grounds language: The critics show that two of the robots used in the experiment achieve a lower prediction error than the others. Thus, those two robots are safer, according to our definition, since they ground the proposed command more accurately.
[1]
R. Rosenfeld.
Nature
,
2009,
Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.
[2]
Antoine Cully,et al.
Robots that can adapt like animals
,
2014,
Nature.
[3]
Joshua Bongard,et al.
Collective Design of Robot Locomotion
,
2014,
ALIFE.
[4]
Josh Bongard,et al.
Robots can ground crowd-proposed symbols by forming theories of group mind
,
2016,
ALIFE.
[5]
Cynthia Breazeal,et al.
Crowdsourcing human-robot interaction
,
2013,
HRI 2013.
[6]
L. Christophorou.
Science
,
2018,
Emerging Dynamics: Science, Energy, Society and Values.
[7]
Sonia Chernova,et al.
Crowdsourcing the construction of a 3D object recognition database for robotic grasping
,
2014,
2014 IEEE International Conference on Robotics and Automation (ICRA).
[8]
Eliezer Yudkowsky,et al.
Friendly Artificial Intelligence
,
2012
.
[9]
Kenneth O. Stanley,et al.
Exploiting Open-Endedness to Solve Problems Through the Search for Novelty
,
2008,
ALIFE.
[10]
Anca D. Dragan,et al.
Cooperative Inverse Reinforcement Learning
,
2016,
NIPS.