Development of a monocular vision system for robotic drilling

Robotic drilling for aerospace structures demands a high positioning accuracy of the robot, which is usually achieved through error measurement and compensation. In this paper, we report the development of a practical monocular vision system for measurement of the relative error between the drill tool center point (TCP) and the reference hole. First, the principle of relative error measurement with the vision system is explained, followed by a detailed discussion on the hardware components, software components, and system integration. The elliptical contour extraction algorithm is presented for accurate and robust reference hole detection. System calibration is of key importance to the measurement accuracy of a vision system. A new method is proposed for the simultaneous calibration of camera internal parameters and hand-eye relationship with a dedicated calibration board. Extensive measurement experiments have been performed on a robotic drilling system. Experimental results show that the measurement accuracy of the developed vision system is higher than 0.15 mm, which meets the requirement of robotic drilling for aircraft structures.

[1]  Gary M. Bone,et al.  Vision-guided fixtureless assembly of automotive components , 2003 .

[2]  José María Martínez-Otzeta,et al.  Particle Filtering for Industrial 6DOF Visual Servoing , 2014, J. Intell. Robotic Syst..

[3]  John E. Inman,et al.  ONCE (ONe-sided Cell End effector) Robotic Drilling System , 2002 .

[4]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[5]  Not Available Not Available,et al.  In this issue , 2002, Clinical Autonomic Research.

[6]  Michael G. Strintzis,et al.  Stereo vision system for precision dimensional inspection of 3D holes , 2003, Machine Vision and Applications.

[7]  John Hartmann,et al.  Flex Track for use in Production , 2005 .

[8]  Qiang Zhan,et al.  Hand–eye calibration and positioning for a robot drilling system , 2012 .

[9]  Mats Björkman,et al.  Cost-efficient drilling using industrial robots with high-bandwidth force feedback , 2010, Robotics and Computer-Integrated Manufacturing.

[10]  Hakan Bilen,et al.  Developing robust vision modules for microsystems applications , 2010, Machine Vision and Applications.

[11]  José Manuel Pastor,et al.  Statistical Pattern Modeling in Vision-Based Quality Control Systems , 2003, J. Intell. Robotic Syst..

[12]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[13]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[14]  Mubarak Shah,et al.  A Fast algorithm for active contours and curvature estimation , 1992, CVGIP Image Underst..

[15]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[16]  Weidong Zhu,et al.  An off-line programming system for robotic drilling in aerospace manufacturing , 2013 .

[17]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  José Mauricio S. T. Motta,et al.  Robot calibration using a 3D vision-based measurement system with a single camera , 2001 .

[19]  Jun Lu,et al.  Virtual manipulator-based binocular stereo vision positioning system and errors modelling , 2010, Machine Vision and Applications.

[20]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Y. Zhang,et al.  Robotic Welding Systems with Vision-Sensing and Self-learning Neuron Control of Arc Welding Dynamic Process , 2003, J. Intell. Robotic Syst..

[22]  Fadi Dornaika,et al.  Simultaneous robot-world and hand-eye calibration , 1998, IEEE Trans. Robotics Autom..

[23]  Yu Zhou,et al.  Integrating Optical Force Sensing with Visual Servoing for Microassembly , 2000, J. Intell. Robotic Syst..

[24]  Hugo Vieira Neto,et al.  Real-time Automated Visual Inspection using Mobile Robots , 2007, J. Intell. Robotic Syst..

[25]  Koji Nakano,et al.  Fast Ellipse Detection Algorithm Using Hough Transform on the GPU , 2011, 2011 Second International Conference on Networking and Computing.

[26]  Gerd Hirzinger,et al.  More accurate camera and hand-eye calibrations with unknown grid pattern dimensions , 2008, 2008 IEEE International Conference on Robotics and Automation.

[27]  Roland Siegwart,et al.  Vision Based Position Control for MAVs Using One Single Circular Landmark , 2011, J. Intell. Robotic Syst..

[28]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[29]  Y. S. Tarng,et al.  Surface roughness inspection by computer vision in turning operations , 2001 .

[30]  Zijian Zhao,et al.  A flexible method combining camera calibration and hand-eye calibration , 2013, Robotica.

[31]  Gerd Hirzinger,et al.  On the issue of camera calibration with narrow angular field of view , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Sara Eastwood,et al.  Industrial Robot : An International Journal Automated aerostructure assembly , 2016 .

[33]  John N. Lygouras,et al.  Vision-Based Path Generation Method for a Robot-Based Arc Welding System , 2007, J. Intell. Robotic Syst..

[34]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[35]  Abed Malti,et al.  Hand-eye calibration with epipolar constraints: Application to endoscopy , 2013, Robotics Auton. Syst..