Implicit Feedback between Reconstruction and Tracking in a Combined Optimization Approach

In this work, we present a combined approach to tracking and reconstruction. An implicit feedback of 3d information to the tracking process is achieved by optimizing a single error function, instead of two separate steps. No assumptions about the error distribution of the tracker are needed in the reconstruction step either. This results in higher reconstruction accuracy and improved tracking robustness in our experimental evaluations. The approach is suited for online reconstruction and has a close to real-time performance on current computing hardware.

[1]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[2]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Stefano Soatto,et al.  A semi-direct approach to structure from motion , 2003, The Visual Computer.

[4]  Detecting coplanar feature points in handheld image sequences , 2007, VISAPP.

[5]  Carsten Rother,et al.  Projective factorization of planes and cameras in multiple views , 2002, Object recognition supported by user interaction for service robots.

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[8]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[9]  Olaf Kähler,et al.  Rigid Motion Constraints for Tracking Planar Objects , 2007, DAGM-Symposium.

[10]  Richard Szeliski,et al.  Vision Algorithms: Theory and Practice , 2002, Lecture Notes in Computer Science.

[11]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[12]  David Nister,et al.  Bundle Adjustment Rules , 2006 .

[13]  Nassir Navab,et al.  A real-time tracking system combining template-based and feature-based approaches , 2007, VISAPP.

[14]  Leif Kobbelt,et al.  Iterative multi - view plane fitting , 2006 .

[15]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Reinhard Koch,et al.  Robust GPU-assisted camera tracking using free-form surface models , 2007, Journal of Real-Time Image Processing.