Refining hand-action models through repeated observations of human and robot behavior by combined template matching

Describes research on learning task level representations by a robot through observation of human demonstrations. We focus on human hand actions and develop a construction method of a human task model which integrates multiple observations to solve ambiguity based on attention points (AP). So far, this analysis constructs a symbolic task model efficiently in a coarse-to-fine way through two steps. However, to represent delicate motion appearing in a task, the system must incorporate the information about precise motion of the manipulated objects into an abstract task model. We propose a method to identify the manipulated object through repeated observations of both human and robot behavior. To this end, we present a method which combines 2D and 3D template matching techniques to localize an object in 3D space generated from a depth and an intensity image. We apply this technique to recognition of human and robot behavior by obtaining the precise trajectory of the manipulated objects. We also present the experimental results achieved through the use of a human-form robot equipped with a 9-eye stereo vision system.