Automatic Pose Estimation of Uncalibrated Multi-View Images Based on a Planar Object with a Predefined Contour Model

We have presented a framework to obtain camera pose (i.e., position and orientation in the 3D space) with real scale information of the uncalibrated multi-view images and the intrinsic camera parameters automatically. Our framework consists of two key steps. First, the initial value of the intrinsic camera and the pose parameters were extracted from homography estimation based on the contour model of some planar objects. Second, a refinement of the intrinsic camera and pose parameters was operated by the bundle adjustment procedure. Our framework can provide a complete flow of pose estimation of disorderly or orderly uncalibrated multi-view images, which can be used in vision tasks requiring scale information. Real multi-view images were utilized to demonstrate the robustness, flexibility and accuracy of the proposed framework. The proposed framework was also applied in 3D reconstruction.

[1]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[2]  Éric Marchand,et al.  A robust model-based tracker combining geometrical and color edge information , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Maarten Vergauwen,et al.  3D Recording for Archaeological Fieldwork , 2003, IEEE Computer Graphics and Applications.

[4]  Frank Dellaert,et al.  Incremental light bundle adjustment for robotics navigation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Henrik I. Christensen,et al.  Real-time 3D model-based tracking using edge and keypoint features for robotic manipulation , 2010, 2010 IEEE International Conference on Robotics and Automation.

[6]  Éric Marchand,et al.  Real-time Hybrid Tracking using Edge and Texture Information , 2007, Int. J. Robotics Res..

[7]  Patrick Rives,et al.  Real-time Quadrifocal Visual Odometry , 2010, Int. J. Robotics Res..

[8]  Bolan Jiang Calibration-free Line-based Tracking for Video Augmentation , 2006, CGVR.

[9]  Rafael Grompone von Gioi,et al.  LSD: A Fast Line Segment Detector with a False Detection Control , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Adilson Berveglieri,et al.  Automatic Orientation of Multi-Scale Terrestrial Images for 3D Reconstruction , 2014, Remote. Sens..

[11]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[12]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[13]  Minas E. Spetsakis,et al.  A multi-frame approach to visual motion perception , 1991, International Journal of Computer Vision.

[14]  Luc Van Gool,et al.  From images to 3D models , 2002, CACM.

[15]  Richard Bowden,et al.  Mutual Information for Lucas-Kanade Tracking (MILK): An Inverse Compositional Formulation , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Maarten Vergauwen,et al.  Web-based 3D Reconstruction Service , 2006, Machine Vision and Applications.

[17]  Rune Havnung Bakken,et al.  Semi-automatic Camera Calibration Using Coplanar Control Points , 2009 .

[18]  Jessica Fuerst Mathematical Methods In Computer Vision , 2016 .

[19]  V. Indelman,et al.  Probabilistic analysis of incremental light bundle adjustment , 2013, 2013 IEEE Workshop on Robot Vision (WORV).

[20]  Éric Marchand,et al.  Accurate real-time tracking using mutual information , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[21]  D. Nistér Automatic passive recovery of 3D from images and video , 2004 .

[22]  Xiaoping Yun,et al.  Design, Implementation, and Experimental Results of a Quaternion-Based Kalman Filter for Human Body Motion Tracking , 2006, IEEE Trans. Robotics.

[23]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[24]  Zhu Bin Determining Intrinsic and Pose Parameters of Camera Based on Concentric Circles , 2010, 2010 International Conference on Digital Manufacturing & Automation.

[25]  Pascal Monasse,et al.  UNORDERED FEATURE TRACKING MADE FAST AND EASY , 2011 .

[26]  John Oliensis,et al.  A Multi-Frame Structure-from-Motion Algorithm under Perspective Projection , 1999, International Journal of Computer Vision.

[27]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[28]  K. Kanatani,et al.  Bundle Adjustment for 3-D Reconstruction: Implementation and Evaluation , 2011 .

[29]  Andrea Fusiello,et al.  Structure-and-motion pipeline on a hierarchical cluster tree , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[30]  Patrick Rives,et al.  An Efficient Direct Approach to Visual SLAM , 2008, IEEE Transactions on Robotics.

[31]  Roberto Cipolla,et al.  Visual tracking and control using Lie algebras , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[32]  Yang Cheng,et al.  Vision-based localization in urban environments , 2005, SPIE Defense + Commercial Sensing.

[33]  Simon Lacroix,et al.  Monocular-vision based SLAM using Line Segments , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[34]  Frank Dellaert,et al.  Incremental Light Bundle Adjustment , 2012, BMVC.

[35]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[36]  M. Pierrot Deseilligny,et al.  APERO, AN OPEN SOURCE BUNDLE ADJUSMENT SOFTWARE FOR AUTOMATIC CALIBRATION AND ORIENTATION OF SET OF IMAGES , 2012 .

[37]  Nassir Navab,et al.  A Unified Approach Combining Photometric and Geometric Information for Pose Estimation , 2008, BMVC.

[38]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Ian D. Reid,et al.  Adaptive relative bundle adjustment , 2009, Robotics: Science and Systems.

[40]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[41]  Marcello R. Napolitano,et al.  Comparison of point matching algorithms for the UAV aerial refueling problem , 2010, Machine Vision and Applications.

[42]  David Nister,et al.  Bundle Adjustment Rules , 2006 .

[43]  Volkan Atalay,et al.  Silhouette-based 3-D model reconstruction from multiple images , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[44]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[45]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Zhengyou Zhang,et al.  Incremental Motion Estimation Through Local Bundle Adjustment , 2001 .

[47]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[48]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[49]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[50]  Michel Dhome,et al.  Generic and real-time structure from motion using local bundle adjustment , 2009, Image Vis. Comput..

[51]  Pascal Fua,et al.  Dynamic and scalable large scale image reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[52]  Qifeng Yu,et al.  Contour model based homography estimation of texture-less planar objects in uncalibrated images , 2016, Pattern Recognit..

[53]  Emanuele Frontoni,et al.  Robot localization in urban environments using omnidirectional vision sensors and partial heterogeneous apriori knowledge , 2010, Proceedings of 2010 IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications.

[54]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[55]  Michel Dhome,et al.  Monocular Vision for Mobile Robot Localization and Autonomous Navigation , 2007, International Journal of Computer Vision.

[56]  Robert M. Haralick,et al.  Analysis and solutions of the three point perspective pose estimation problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[57]  Irfan A. Essa,et al.  Propagation of innovative information in non-linear least-squares structure from motion , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[58]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[59]  Fabio Remondino,et al.  Orientation and 3D modelling from markerless terrestrial images: combining accuracy with automation , 2010 .

[60]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[61]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[62]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[63]  Frank Dellaert,et al.  iSAM2: Incremental smoothing and mapping using the Bayes tree , 2012, Int. J. Robotics Res..

[64]  Philip David,et al.  Object recognition in high clutter images using line features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[65]  Simon Fuhrmann,et al.  MVE - A Multi-View Reconstruction Environment , 2014, GCH.

[66]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[67]  Paolo Pirjanian,et al.  The vSLAM Algorithm for Robust Localization and Mapping , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[68]  Michael Goesele,et al.  Let There Be Color! Large-Scale Texturing of 3D Reconstructions , 2014, ECCV.

[69]  Chunming Li,et al.  Distance Regularized Level Set Evolution and Its Application to Image Segmentation , 2010, IEEE Transactions on Image Processing.

[70]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[71]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[72]  S. Del Pizzo,et al.  AUTOMATIC ORIENTATION OF IMAGE SEQUENCES IN CULTURAL HERITAGE , 2012 .

[73]  Maxime Lhuillier,et al.  Automatic scene structure and camera motion using a catadioptric system , 2008, Comput. Vis. Image Underst..

[74]  Ian D. Reid,et al.  RSLAM: A System for Large-Scale Mapping in Constant-Time Using Stereo , 2011, International Journal of Computer Vision.

[75]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.