Force control and visual servoing using planar surface identification

When designing flexible multi-sensor based robot systems, one important problem is how to combine the measurements from disparate sensors such as cameras and force sensors. In this paper, we present a method for combining direct force control and visual servoing in the presence of unknown planar surfaces. The control algorithm involves a force feedback control loop and a vision based reference trajectory as a feed-forward signal. The vision system is based on a constrained image-based visual servoing algorithm designed for surface following, where the location and orientation of the planar constraint surface is estimated online using position-, force- and visual data. We show how data from a simple and efficient camera calibration method can be used in combination with force and position data to improve the estimation and reference trajectories. The method is validated through experiments involving force controlled drawing on an unknown surface. The robot will grasp a pen and use it to draw lines between a number of markers drawn on a white-board, while the contact force is kept constant. Despite its simplicity, the performance of the method is satisfactory.