Calibration routine for a telecentric stereo vision system considering affine mirror ambiguity

Abstract. A robust calibration approach for a telecentric stereo camera system for three-dimensional (3-D) surface measurements is presented, considering the effect of affine mirror ambiguity. By optimizing the parameters of a rigid body transformation between two marker planes and transforming the two-dimensional (2-D) data into one coordinate frame, a 3-D calibration object is obtained, avoiding high manufacturing costs. Based on the recent contributions in the literature, the calibration routine consists of an initial parameter estimation by affine reconstruction to provide good start values for a subsequent nonlinear stereo refinement based on a Levenberg–Marquardt optimization. To this end, the coordinates of the calibration target are reconstructed in 3-D using the Tomasi–Kanade factorization algorithm for affine cameras with Euclidean upgrade. The reconstructed result is not properly scaled and not unique due to affine ambiguity. In order to correct the erroneous scaling, the similarity transformation between one of the 2-D calibration plane points and the corresponding 3-D points is estimated. The resulting scaling factor is used to rescale the 3-D point data, which then allows in combination with the 2-D calibration plane data for a determination of the start values for the subsequent nonlinear stereo refinement. As the rigid body transformation between the 2-D calibration planes is also obtained, a possible affine mirror ambiguity in the affine reconstruction result can be robustly corrected. The calibration routine is validated by an experimental calibration and various plausibility tests. Due to the usage of a calibration object with metric information, the determined camera projection matrices allow for a triangulation of correctly scaled metric 3-D points without the need for an individual camera magnification determination.

[1]  T. Kanade,et al.  Perspective Factorization Methods for Euclidean Reconstruction , 1999 .

[2]  Peisen S. Huang,et al.  Novel method for structured light system calibration , 2006 .

[3]  Qian Chen,et al.  A new microscopic telecentric stereo vision system - Calibration, rectification, and three-dimensional reconstruction , 2019, Optics and Lasers in Engineering.

[4]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[5]  Sami S. Brandt,et al.  A Bayesian Approach for Affine Auto-calibration , 2005, SCIA.

[6]  Haibo Liu,et al.  A Flexible Calibration Approach for Cameras with Double-Sided Telecentric Lenses , 2016 .

[7]  D. C. Brown,et al.  Lens distortion for close-range photogrammetry , 1986 .

[8]  Dong Li,et al.  An accurate calibration method for a camera with telecentric lenses , 2013 .

[9]  Long Quan,et al.  Self-calibration of an affine camera from multiple views , 1996, International Journal of Computer Vision.

[10]  Feng Gao,et al.  A simple, flexible and automatic 3D calibration method for a phase calculation-based fringe projection imaging system. , 2013, Optics express.

[11]  J J Koenderink,et al.  Affine structure from motion. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[12]  Stéphane Christy,et al.  Euclidean reconstruction and affine camera calibration using controlled robot motions , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[13]  Luc Van Gool,et al.  Multibody Structure-from-Motion in Practice , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Adrien Bartoli,et al.  Planar Structure-from-Motion with Affine Camera Models: Closed-Form Solutions, Ambiguities and Degeneracy Analysis , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Adrien Bartoli,et al.  Affine Approximation for Direct Batch Recovery of Euclidian Structure and Motion from Sparse Data , 2006, International Journal of Computer Vision.

[16]  Song Zhang,et al.  High-speed 3D shape measurement with structured light methods: A review , 2018, Optics and Lasers in Engineering.

[17]  Xianmin Zhang,et al.  Calibration of camera with small FOV and DOF telecentric lens , 2013, ROBIO.

[18]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Douglas Lanman,et al.  Shape from depth discontinuities under orthographic projection , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[20]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[21]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[22]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[23]  Haibo Liu,et al.  Epipolar rectification method for a stereovision system with telecentric cameras , 2016 .

[24]  Gang Xu,et al.  A Unified Theory of Uncalibrated Stereo for Both Perspective and Affine Cameras , 1998, Journal of Mathematical Imaging and Vision.

[25]  Kenichi Kanatani,et al.  Guide to 3D Vision Computation , 2016, Advances in Computer Vision and Pattern Recognition.

[26]  Xiang Peng,et al.  Distortion correction for microscopic fringe projection system with Scheimpflug telecentric lens. , 2015, Applied optics.

[27]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[28]  Eduard Reithmeier,et al.  Accurate calibration of a fringe projection system by considering telecentricity , 2011, Optical Metrology.

[29]  Hui Lin,et al.  Structure light telecentric stereoscopic vision 3D measurement system based on Scheimpflug condition , 2016 .

[30]  Qian Chen,et al.  Calibration of telecentric cameras with distortion center estimation , 2018, Other Conferences.

[31]  Qiang Liu,et al.  Microscopic Three-Dimensional Measurement Based on Telecentric Stereo and Speckle Projection Methods , 2018, Sensors.

[32]  Heming Huang,et al.  Flexible calibration method for telecentric fringe projection profilometry systems. , 2016, Optics express.

[33]  Zhong Chen,et al.  Telecentric stereo micro-vision system: Calibration method and experiments , 2014 .

[34]  Sam Van der Jeught,et al.  Real-time structured light profilometry: a review , 2016 .

[35]  S. Ullman.  The interpretation of structure from motion , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[36]  Ehud Rivlin,et al.  A Geometric Interpretation of Weak-Perspective Motion , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Beiwen Li,et al.  Flexible calibration method for microscopic structured light system using telecentric lens. , 2015, Optics express.

[38]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[39]  Robert B. Fisher,et al.  Estimating 3-D rigid body transformations: a comparison of four major algorithms , 1997, Machine Vision and Applications.

[40]  Dong Li,et al.  Telecentric 3D profilometry based on phase-shifting fringe projection. , 2014, Optics express.

[41]  Friedel Koerfer,et al.  Optisches Messen technischer Oberflächen : Messprinzipien und Begriffe , 2009 .

[42]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[43]  Sami S. Brandt,et al.  Conditional solutions for the affine reconstruction of N-views , 2005, Image Vis. Comput..

[44]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[45]  Haibo Liu,et al.  Calibration method for projector-camera-based telecentric fringe projection profilometry system. , 2017, Optics express.