Enhancing Robot Perception Using Human Teammates ∗ (Extended Abstract)

In robotics research, perception is one of the most challenging tasks. In contrast to existing approaches that rely only on computer vi- sion, we propose an alternative method for improving perception by learning from human teammates. To evaluate, we apply this idea to a door detection problem. A set of preliminary experiments has been completed using software agents with real vision data. Our results demonstrate that information inferred from teammate observations significantly improves the perception precis ion. 1. BACKGROUND Robot perception is generally formulated as a problem of ana- lyzing and interpreting various sensory inputs, e.g., camera feeds. In this paper, we approach robot perception from a completely dif- ferent direction. Our approach utilizes a team setting where a robot collaborates with human teammates. Motivated by the fact that humans possess superior perception skills relative to thei r robotic counterparts, we investigate how a robot can take advantage of its teammate's perfect vision. In general, an agent acquires new information through percep- tion, and in turn, the agent chooses actions based on the informa- tion acquired. Let us suppose that a robot has a mental model of its human teammate such that a causal relationship is specifi ed be- tween information and actions. Then, by understanding the human mental model of such decision making (or planning), the robot can infer what the human teammate has seen based on the human's be- havior. In other words, an observation of a human teammate can be ∗ This work was conducted (in part) through collaborative partic- ipation in the Robotics Consortium sponsored by the U.S Army