Functional Descriptors for Object Affordances

In the context of robot learning from demonstration, it is very important that a robot understands what an object can be used for. By observing a human performing an activity, a robot should be able to identify the human motion, the objects involved and the outcome of the performed activity. One important aspect of this challenging problem is to detect and reason about objects in terms of affordance or alternatively, about their function in the current activity. Affordance is often modeled in terms of appearance however appearance does not necessarily map one-to-one with functional classes. In this paper we propose two alternative features that characterize objects directly in terms of how they are used. Our approach show a significant improvement compared to the traditional appearance based methods.