Learning dextrous manipulation skills using multisensory information

We present a method for autonomous learning of dextrous manipulation skills with multifingered robot hands. We use heuristics derived from observations made on human hands to reduce the degrees of freedom of the task and make learning tractable. Our approach consists of learning and storing a few basic manipulation primitives for a few prototypical objects and then using an associative memory to obtain the required parameters for new objects and/or manipulations. During learning, sensory information from tactile sensors and a position measuring device is used to evaluate the quality of a candidate manipulation. The parameter space of the robot is searched using a modified version of the evolution strategy, which is robust to the noise normally present in real-world complex robotic tasks. To ensure that the learned skills are applicable in the real world, our system does not rely on simulation. Experimental results show that accurate dextrous manipulation skills can be learned by the robot in a short period of time.