Tracking and Reconstruction in a Combined Optimization Approach

We present a novel approach to the structure-from-motion problem which combines the search for correspondences and geometric reconstruction, rather than treating these as separate steps. Through the combination of the two steps, we achieve an implicit feedback of 3D information to aid the correspondence search, and at the same time we avoid an explicit model for tracking errors. The reconstruction results are therefore optimal in case of, for example, Gaussian noise on image intensities. We also present an efficient online framework for structure-from-motion with our combined approach, thoroughly evaluate the method in experiments and compare the results to state-of-the-art methods.

[1]  Stefano Soatto,et al.  Optimal Structure from Motion: Local Ambiguities and Global Estimates , 2004, International Journal of Computer Vision.

[2]  Thomas S. Huang,et al.  Theory of Reconstruction from Image Motion , 1992 .

[3]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Axel Pinz,et al.  Robust Pose Estimation from a Planar Target , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Nassir Navab,et al.  A real-time tracking system combining template-based and feature-based approaches , 2007, VISAPP.

[6]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[7]  Stefano Soatto,et al.  A semi-direct approach to structure from motion , 2003, The Visual Computer.

[8]  Carsten Rother,et al.  Projective factorization of planes and cameras in multiple views , 2002, Object recognition supported by user interaction for service robots.

[9]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  K. Ikeuchi,et al.  Iterative Estimation of Rotation and Translation using the Quaternion , 1995 .

[11]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[12]  Leif Kobbelt,et al.  Iterative multi - view plane fitting , 2006 .

[13]  Dana Cobzas,et al.  3D SSD tracking with estimated 3D planes , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[14]  Patrick Rives,et al.  Constrained Multiple Planar Template Tracking for Central Catadioptric Cameras , 2006, BMVC.

[15]  Bill Triggs,et al.  Plane+Parallax, Tensors and Factorization , 2000, ECCV.

[16]  Patrick Rives,et al.  An Efficient Direct Approach to Visual SLAM , 2008, IEEE Transactions on Robotics.

[17]  Olaf Kähler,et al.  Implicit Feedback between Reconstruction and Tracking in a Combined Optimization Approach , 2008, DAGM-Symposium.

[18]  J. Salz,et al.  Algorithms for estimation of three-dimensional motion , 1985, AT&T Technical Journal.

[19]  David Nister,et al.  Bundle Adjustment Rules , 2006 .

[20]  Amnon Shashua,et al.  The Rank 4 Constraint in Multiple (>=3) View Geometry , 1996, ECCV.

[21]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[22]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[23]  Cristian Sminchisescu,et al.  Building Roadmaps of Minima and Transitions in Visual Models , 2004, International Journal of Computer Vision.

[24]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[25]  Detecting coplanar feature points in handheld image sequences , 2007, VISAPP.

[26]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[27]  Loong Fah Cheong,et al.  Understanding the Behavior of SFM Algorithms: A Geometric Approach , 2004, International Journal of Computer Vision.

[28]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Olaf Kähler,et al.  Robust Real-Time SFM in a Combined Formulation of Tracking and Reconstruction , 2008, VMV.

[30]  Reinhard Koch,et al.  Dynamic 3-D Scene Analysis Through Synthesis Feedback Control , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  P. Anandan,et al.  About Direct Methods , 1999, Workshop on Vision Algorithms.

[32]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision , 2004 .

[33]  John Oliensis.  The Least-Squares Error for Structure from Infinitesimal Motion , 2004, International Journal of Computer Vision.

[34]  S. Negahdaripour.  Closed-form relationship between the two interpretations of a moving plane , 1990 .

[35]  Bill Triggs,et al.  Autocalibration from Planar Scenes , 1998, ECCV.

[36]  Lihi Zelnik-Manor,et al.  Multiview Constraints on Homographies , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Olaf Kähler,et al.  Rigid Motion Constraints for Tracking Planar Objects , 2007, DAGM-Symposium.