Task Oriented Model-Driven Visually Servoed Agents

Most proposed visual servoing control strategies use image-based visual servoing techniques in which the visual servoing control loop is closed using 2D image plane coordinates rather than 3D world coordinates. These strategies do not allow for explicit representations of the 3D world. We propose a visual servoing framework that uses 3D environment models to describe the task. These models are augmented by sensor mappings that represent the visual sensors used by the system and are used to drive an image-based visually servoed agent to execute the task in the real world. This framework allows for the use of 3D reasoning within the environment model, while taking advantage of the benefits 2D image-based visual servoing provides. In this paper, we describe our proposed framework and present experimental results which show that the framework can be used to successfully perform visually servoed manipulation tasks.