Transferring Category-Based Functional Grasping Skills by Latent Space Non-Rigid Registration

Objects within a category are often similar in their shape and usage. When we—as humans—want to grasp something, we transfer our knowledge from past experiences and adapt it to novel objects. In this letter, we propose a new approach for transferring grasping skills that accumulates grasping knowledge into a category-level canonical model. Grasping motions for novel instances of the category are inferred from geometric deformations between the observed instance and the canonical shape. Correspondences between the shapes are established by means of a non-rigid registration method that combines the coherent point drift approach with subspace methods. By incorporating category-level information into the registration, we avoid unlikely shapes and focus on deformations actually observed within the category. Control poses for generating grasping motions are accumulated in the canonical model from grasping definitions of known objects. According to the estimated shape parameters of a novel instance, the control poses are transformed toward it. The category-level model makes our method particularly relevant for online grasping, where fully observed objects are not easily available. This is demonstrated through experiments in which objects with occluded handles are successfully grasped.