Automatic dismantling integrating optical flow into a machine vision-controlled robot system

In order to automatically run a dismantling task, first a workpiece has to be recognized and its pose has to be estimated. This step is solved by a model-based recognition process using the information from three stationary cameras. Our workpiece model could be extracted from CAD-data and consists of planar patches and circular features, where the circular features are suitable to model parts of connection elements like screws. Using the initial estimation of the workpiece pose we control the robot by evaluating the image sequence given by a camera mounted on the robot hand. We use a multi-level 3D tracking approach to continuously estimate the workpiece pose using an iterative extended Kalman-filter process and integrate velocity estimation based on optical flow. To provide enough computing power for real-time image processing, we combine our dedicated image processing hardware with a heterogeneous parallel computer. Our approach has been tested in numerous experiments on loosening screws with a powered screwdriver and dismantling a workpiece consisting of two blocks of different sizes connected by two screws.

[1]  Antti J. Koivo,et al.  Real-time vision feedback for servoing robotic manipulator with self-tuning controller , 1991, IEEE Trans. Syst. Man Cybern..

[2]  Paolo Dario,et al.  An investigation on a robot system for disassembly automation , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[3]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[4]  Hans-Hellmut Nagel,et al.  Multi-Level 3D-Tracking of Objects Integrating Velocity Estimation based on Optical Flow and Kalman-Filtering , 1995, ISER.

[5]  Akio Kosaka,et al.  Vision-based motion tracking of frigid objects using prediction of uncertainties , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[6]  Roger Y. Tsai,et al.  A new technique for fully autonomous and efficient 3D robotics hand/eye calibration , 1988, IEEE Trans. Robotics Autom..

[7]  Peter K. Allen,et al.  Active, uncalibrated visual servoing , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[8]  Gregory D. Hager,et al.  Robot feedback control based on stereo vision: towards calibration-free hand-eye coordination , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[9]  Tony P. Pridmore,et al.  TINA: a 3D vision system for pick and place , 1988, Image Vis. Comput..

[10]  C. S. George Lee,et al.  Model-based visual feedback control for a hand-eye coordinated robotic system , 1992, Computer.

[11]  Tsutomu Kimoto,et al.  Manipulator control with image-based visual servo , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[12]  W. J. Wilson,et al.  Visual Servo Control of Robots Using Kalman Filter Estimates of Relative Pose , 1993 .

[13]  Thomas Skordas,et al.  Calibrating a mobile camera , 1990, Image Vis. Comput..