Multi-cameras visual servoing for dual-arm coordinated manipulation

Although image-based visual servoing (IBVS) provides good performance in many dual-arm manipulation applications, it reveals some fatal limitations when dealing with a large position and orientation uncertainty. The object features may leave the camera's field of view, and the dual-arm robot may not converge to their goal configurations. In this paper, a novel vision-based control strategy is presented to resolve these limitations. A visual path planning method for dual-arm end-effector features is proposed to regulate the large initial poses to the pre-alignment poses. Then, the visual constraints between the position and orientation of two objects are established, and the sequenced subtasks are performed to attain the pose alignment of two objects by using a multi-tasks IBVS method. The proposed strategy has been implemented on a MOTOMAN robot to perform the alignment tasks of plug–socket and cup–lid, and results indicate that the plug and socket with the large initial pose errors 145.4 mm, 43.8○ (the average errors of three axes) are successfully aligned with the allowed pose alignment errors 3.1 mm, 1.1○, and the cup and lid with the large initial pose errors 131.7 mm, 20.4○ are aligned with the allowed pose alignment errors −2.7 mm, −0.8○.