Automated initialization for marker-less tracking: a sensor fusion approach

We introduce a sensor fusion approach for automated initialization of marker-less tracking systems. It is not limited in tracking range and working environment, given a 3D model of the objects or the real scene. This is achieved based on a statistical analysis and probabilistic estimation of the uncertainties of the tracking sensors. The explicit representation of the error distribution allows the fusion of different sensor data. This methodology was applied to an augmented reality system, using a mobile camera and several stationary tracking sensors, and can be easily extended to the case of any additional sensor. In order to solve the initialization problem, we adapt, modify and integrate advanced techniques such as plenoptic viewing, intensity-based registration, and ICP. Thereby, the registration error is minimized in 3D object space rather than in 2D image. Experimental results show how complex objects can be registered efficiently and accurately to a single image.

[1]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[2]  Xinhua Zhuang,et al.  Pose estimation from corresponding point data , 1989, IEEE Trans. Syst. Man Cybern..

[3]  Yonghuai Liu,et al.  Improving ICP with easy implementation for free-form surface matching , 2004, Pattern Recognit..

[4]  Vincent Lepetit,et al.  Stable real-time 3D tracking using online and offline information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Hideyuki Tamura,et al.  Robot vision-based registration utilizing bird's-eye view with user's view , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[6]  Suya You,et al.  Natural Feature Tracking for Augmented Reality , 1999, IEEE Trans. Multim..

[7]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[8]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[9]  Yakup Genc,et al.  Marker-less tracking for AR: a learning-based approach , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[10]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[11]  Olivier Faugeras,et al.  Three-Dimensional Computer Vision , 1993 .

[12]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[13]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[14]  Dieter Schmalstieg,et al.  Fundamentals of Ubiquitous Tracking , 2004 .

[15]  Gudrun Klinker,et al.  Model-based tracking with stereovision for AR , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[16]  Gerd Hirzinger,et al.  Registration of CAD-models to images by iterative inverse perspective matching , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[17]  Karl-Rudolf Koch,et al.  Parameter estimation and hypothesis testing in linear models , 1988 .

[18]  Gregory D. Hager,et al.  Fast and Globally Convergent Pose Estimation from Video Images , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Linda G. Shapiro,et al.  Computer and Robot Vision (Volume II) , 2002 .

[20]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[21]  Adrian David Cheok,et al.  Online 6 DOF augmented reality registration from natural features , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[22]  Tyrone L. Vincent,et al.  Analysis of Head Pose Accuracy in Augmented Reality , 2000, IEEE Trans. Vis. Comput. Graph..

[23]  Katsushi Ikeuchi,et al.  Simultaneous 2D images and 3D geometric model registration for texture mapping utilizing reflectance attribute , 2002 .

[24]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[25]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[26]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Yonghuai Liu,et al.  3D shape matching using collinearity constraint , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[28]  P. Debevec,et al.  Image-based modeling, rendering, and lighting , 2002, IEEE Computer Graphics and Applications.

[29]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[30]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Vincent Lepetit,et al.  Fully automated and stable registration for augmented reality applications , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[32]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.