3D Motion from structures of points, lines and planes

In this article we propose a method for estimating the camera motion from a video-sequence acquired in the presence of general 3D structures. Solutions to this problem are commonly based on the tracking of point-like features, as they usually back-project onto viewpoint-invariant 3D features. In order to improve the robustness, the accuracy and the generality of the approach, we are interested in tracking and using a wider class of structures. In addition to points, in fact, we also simultaneously consider lines and planes. In order to be able to work on all such structures with a compact and unified formalism, we use here the Conformal Model of Geometric Algebra, which proved very powerful and flexible. As an example of application of our approach, we propose a causal algorithm based on an Extended Kalman Filter, for the estimation of 3D structure and motion from 2D observations of points, lines and coplanar features, and we evaluate its performance on both synthetic and real sequences.

[1]  S. Shankar Sastry,et al.  Rank Conditions on the Multiple-View Matrix , 2004, International Journal of Computer Vision.

[2]  G. Sommer.  Geometric computing with Clifford algebras: theoretical foundations and applications in computer vision and robotics , 2001 .

[3]  Stefano Soatto,et al.  3-D Motion and Structure from 2-D Motion Causally Integrated over Time: Implementation , 2000, ECCV.

[4]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[5]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[6]  Adrien Bartoli,et al.  The 3D Line Motion Matrix and Alignment of Line Reconstructions , 2004, International Journal of Computer Vision.

[7]  Tamar Frankel.  [The theory and the practice...]. , 2001, Tijdschrift voor diergeneeskunde.

[8]  Richard Szeliski,et al.  Geometrically Constrained Structure from Motion: Points on Planes , 1998, SMILE.

[9]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[10]  Stephen Mann,et al.  Geometric Algebra: A Computational Framework for Geometrical Applications (Part 2) , 2002, IEEE Computer Graphics and Applications.

[11]  Alex Pentland,et al.  Recursive Estimation of Motion, Structure, and Focal Length , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Mohinder S. Grewal,et al.  Kalman Filtering: Theory and Practice , 1993 .

[13]  Frank Dellaert,et al.  Jacobian images of super-resolved texture maps for model-based motion estimation and tracking , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[14]  Olivier Faugeras,et al.  Motion and Structure from Motion in a piecewise Planar Environment , 1988, Int. J. Pattern Recognit. Artif. Intell..

[15]  Thierry Viéville,et al.  Canonical Representations for the Geometries of Multiple Projective Views , 1996, Comput. Vis. Image Underst..

[16]  R. German.  Sintering theory and practice , 1996 .

[17]  Gerald Sommer,et al.  Geometric Computing with Clifford Algebras , 2001, Springer Berlin Heidelberg.

[18]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[19]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision , 2004 .

[20]  Jacob Ström.  STRUCTURE FROM MOTION OF PLANAR SURFACES (ANALYSIS OF AN EKF BASED APPROACH) , 2001 .

[21]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[22]  Mahesan Niranjan,et al.  Uncertainty in geometric computations , 2002 .

[23]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[24]  Olivier D. Faugeras,et al.  Feed-forward recovery of motion and structure from a sequence of 2D-lines matches , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[25]  A. Pentland,et al.  Real time tracking and modeling of faces: an EKF-based analysis by synthesis approach , 1999, Proceedings IEEE International Workshop on Modelling People. MPeople'99.

[26]  Eduardo Bayro-Corrochano,et al.  The motor extended Kalman filter for dynamic rigid motion estimation from line observations , 2001 .

[27]  D. Hestenes,et al.  Clifford Algebra to Geometric Calculus: A Unified Language for Mathematics and Physics , 1984 .

[28]  Stan Sclaroff,et al.  Recursive estimation of motion and planar structure , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[29]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Bodo Rosenhahn,et al.  Pose Estimation in Conformal Geometric Algebra Part II: Real-Time Pose Estimation Using Extended Feature Concepts , 2005, Journal of Mathematical Imaging and Vision.

[31]  Stefano Soatto,et al.  Structure from Motion Causally Integrated Over Time , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Y. Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, CVPR 2004.

[33]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[34]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[35]  Eduardo Bayro-Corrochano,et al.  Geometric algebra: a framework for computing point and line correspondences and projective structure using n uncalibrated cameras , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[36]  David J. Kriegman,et al.  Structure and Motion from Line Segments in Multiple Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Joan Lasenby,et al.  Conformal Geometry, Euclidean Space and Geometric Algebra , 2002, ArXiv.

[38]  S. Shankar Sastry,et al.  Multiple view motion estimation and control for landing an unmanned aerial vehicle , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[39]  Joan Lasenby,et al.  Surface Evolution and Representation using Geometric Algebra , 2000, IMA Conference on the Mathematics of Surfaces.

[40]  Jean-Luc Jezouin,et al.  3D structure from a monocular sequence of images , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[41]  Ian D. Reid,et al.  Real-Time Monocular SLAM with Straight Lines , 2006, BMVC.

[42]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Anders Heyden,et al.  Euclidean reconstruction from image sequences with varying and unknown focal length and principal point , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Stephen Mann,et al.  Geometric Algebra: A computational framework for geometrical applications Part 1 , 2002, IEEE Computer Graphics and Applications.

[45]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[46]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[47]  Takeo Kanade,et al.  A unified factorization algorithm for points, line segments and planes with uncertainty models , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[48]  R. Carter.  Lie Groups , 1970, Nature.

[49]  B. Hall.  Lie Groups, Lie Algebras, and Representations , 2003 .

[50]  Nicholas Ayache,et al.  Three-dimensional structure from a monocular sequence of images , 1990 .

[51]  Leo Dorst,et al.  Geometric algebra: A computational framework for geometrical applications (Part I: Algebra) , 2002 .

[52]  Adrien Bartoli,et al.  Constrained Structure and Motion From Multiple Uncalibrated Views of a Piecewise Planar Scene , 2003, International Journal of Computer Vision.

[53]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[54]  Alex Pentland,et al.  3D structure from 2D motion , 1999, IEEE Signal Process. Mag..