Robust Real-Time SFM in a Combined Formulation of Tracking and Reconstruction

Recently it was observed, that a combined formulation of tracking and reconstruction increases the robustness and accuracy of both these steps in structure-from-motion problems [9]. However, the benefits come at the cost of a higher computational complexity. In this work, we present strategies for an efficient implementation of such a combined approach. We identify the time consuming steps in the system and analyze opportunities for simplifying and parallelizing the original problem. An evaluation of the overall system is presented and we show, that frame rates of 5 fps and beyond are achieved on current hardware, without significant losses in robustness and accuracy.

[1]  Olaf Kähler,et al.  Implicit Feedback between Reconstruction and Tracking in a Combined Optimization Approach , 2008, DAGM-Symposium.

[2]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[3]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Barbara Chapman,et al.  Using OpenMP - portable shared memory parallel programming , 2007, Scientific and engineering computation.

[6]  David Nister,et al.  Bundle Adjustment Rules , 2006 .

[7]  Carsten Rother,et al.  Projective factorization of planes and cameras in multiple views , 2002, Object recognition supported by user interaction for service robots.

[8]  Dana Cobzas,et al.  3D SSD tracking with estimated 3D planes , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[9]  Nassir Navab,et al.  A real-time tracking system combining template-based and feature-based approaches , 2007, VISAPP.

[10]  Leif Kobbelt,et al.  Iterative multi - view plane fitting , 2006 .

[11]  Lihi Zelnik-Manor,et al.  Multiview Constraints on Homographies , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[13]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[14]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.