Multisensor fusion of touch and vision using minimal representation size

Multisensor fusion has emerged as a central problem in the development of robotic systems where interaction with the environment is critical to the achievement of a given task. The Anthrobot five-fingered hand grasps an object, and senses the contact points with the surface of the object using tactile sensors. The tactile sensors extract touch position and approximate surface normal in the kinematic reference frame of the hand. In addition, a CCD camera views the position of the same object and extracts vertex/edge features of the object image. Both the tactile features and the visual features are related to the position and orientation of the object, and in practice we wish to combine these two sources of information to improve robot's ability to accurately manipulate the object. The fusion of the tactile and image feature data is used to derive an improved estimate of the object pose which guides the manipulation.