Computer vision issues during eye-in-hand robotic tasks

Our previous work has introduced the framework of controlled active vision that provides a bridge between computer vision and control theory. This paper presents some of the computer vision techniques that were employed in order to detect targets, automatically select features, measure feature displacements, and evaluate measurements during the various applications of the controlled active vision framework. We experimented with a lot of different techniques, but the most robust proved to be the sum-of-squared differences (SSD) optical flow technique. Search optimizations and dynamic pyramiding are proposed in order to provide real-time performance. In addition, several techniques for the evaluation of the measurements are presented. One important characteristic of these techniques is that they can also be used for the selection of features for tracking in conjunction with several numerical criteria that guarantee the robustness of the servoing. These techniques are important aspects of our work since they can be implemented either on-line or off-line. The paper concludes with results from the application of these techniques to real images.

[1]  David Heeger,et al.  Depth and Flow From Motion Energy , 1986, AAAI.

[2]  Joachim Heel,et al.  Dynamic Motion Vision , 1989, Other Conferences.

[3]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[4]  James M. Rehg,et al.  Visual tracking with deformation models , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[5]  P. K. Khosla,et al.  Adaptive Robotic Visual Tracking , 1991, 1991 American Control Conference.

[6]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[7]  Nikolaos Papanikolopoulos Controlled active vision , 1992 .

[8]  Takeo Kanade,et al.  Vision and control techniques for robotic visual tracking , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[9]  Thomas S. Huang,et al.  Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surfaces , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.