Structure and motion from line correspondences: Representation, projection, initialization and sparse bundle adjustment

We address the problem of structure and motion from line correspondences, which ranges from the representation of lines, their projections and the initialization procedure to the final adjustment. The Cayley representation of spatial lines is developed, which is a nonlinear minimal parametrization circumventing the tiresome Plucker constraint. The relationships between different line representations are given. Based on these relationships, we derive a novel line projection function which is consistent with the previous results. After building the line observation model, we employ a closed-form solution for the first image triplet, then develop an incremental initialization approach to initialize the motion and structure parameters. Finally, the sparse bundle adjustment (SBA) is applied to refine the parameters, which updates the spatial lines by using the Cayley representation with an unconstrained optimization engine. The experiments show that the proposed algorithm outperforms the previous works both in efficiency and accuracy.

[1]  Stergios I. Roumeliotis,et al.  Globally optimal pose estimation from line correspondences , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Kenneth S. Roberts,et al.  A new representation for a line , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Reinhard Koch,et al.  An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency , 2013, J. Vis. Commun. Image Represent..

[4]  Tom Drummond,et al.  Edge landmarks in monocular SLAM , 2009, Image Vis. Comput..

[5]  Luis Montano,et al.  Structure and motion from straight line segments , 2000, Pattern Recognit..

[6]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Andrea Fusiello,et al.  Structure-and-motion pipeline on a hierarchical cluster tree , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[8]  Zhanyi Hu,et al.  MSLD: A robust descriptor for line matching , 2009, Pattern Recognit..

[9]  Thomas S. Huang,et al.  Determining 3-D motion and structure of a rigid body using straight line correspondences , 1983, ICASSP.

[10]  Narendra Ahuja,et al.  Motion and Structure from Line Correspondences; Closed-Form Solution, Uniqueness, and Optimization , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Jan Mayer,et al.  A numerical evaluation of preprocessing and ILU-type preconditioners for the solution of unsymmetric sparse linear systems using iterative methods , 2009, TOMS.

[12]  Olivier D. Faugeras,et al.  Feed-forward recovery of motion and structure from a sequence of 2D-lines matches , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[13]  Adrien Bartoli,et al.  The 3D Line Motion Matrix and Alignment of Line Reconstructions , 2004, International Journal of Computer Vision.

[14]  Reinhard Koch,et al.  Hand-Held Monocular SLAM Based on Line Segments , 2011, 2011 Irish Machine Vision and Image Processing Conference.

[15]  Teresa A. Vidal-Calleja,et al.  Impact of Landmark Parametrization on Monocular EKF-SLAM with Points and Lines , 2011, International Journal of Computer Vision.

[16]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Tomás Pajdla,et al.  Line reconstruction from many perspective images by factorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  David J. Kriegman,et al.  Structure and Motion from Line Segments in Multiple Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Minas E. Spetsakis,et al.  Structure from motion using line correspondences , 1990, International Journal of Computer Vision.

[20]  Walterio W. Mayol-Cuevas,et al.  Real-Time Model-Based SLAM Using Line Segments , 2006, ISVC.

[21]  Ian D. Reid,et al.  Real-Time Monocular SLAM with Straight Lines , 2006, BMVC.

[22]  A. Habib,et al.  Bundle Adjustment with Self–Calibration Using Straight Lines , 2002 .

[23]  Yongduek Seo,et al.  Sequential reconstruction of lines in projective space , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[24]  Shree K. Nayar,et al.  Micro Phase Shifting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Lu Wang,et al.  Wide-baseline image matching using Line Signatures , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[26]  D. Hu,et al.  Strapdown inertial navigation system algorithms based on dual quaternions , 2005 .

[27]  Mongi A. Abidi,et al.  Pose and motion estimation from vision using dual quaternion-based extended kalman filtering , 1997 .

[28]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[29]  Steven G. Krantz,et al.  Handbook of Complex Variables , 1999 .

[30]  Shiqi Li,et al.  A Robust O(n) Solution to the Perspective-n-Point Problem , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Reinhard Koch,et al.  Robust and Efficient Pose Estimation from Line Correspondences , 2012, ACCV.

[32]  Takeo Kanade,et al.  A unified factorization algorithm for points, line segments and planes with uncertainty models , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[33]  Adrien Bartoli,et al.  Structure-from-motion using lines: Representation, triangulation, and bundle adjustment , 2005, Comput. Vis. Image Underst..

[34]  James L. Crowley,et al.  Measurement and integration of 3-D structures by tracking edge lines , 1990, International Journal of Computer Vision.

[35]  Johannes Wallner,et al.  Line Geometry for 3D Shape Understanding and Reconstruction , 2004, ECCV.

[36]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[37]  Olivier D. Faugeras,et al.  On the geometry and algebra of the point and line correspondences between N images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[38]  Frank Dellaert,et al.  Line-Based Structure from Motion for Urban Environments , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[39]  Zhanyi Hu,et al.  Line matching leveraged by point correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Xiaoou Tang,et al.  Example-based 3D object reconstruction from line drawings , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Yu Chen,et al.  Decomposition of Complex Line Drawings with Hidden Lines for 3D Planar-Faced Manifold Object Reconstruction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Thomas S. Huang,et al.  Estimation of rigid body motion using straight line correspondences , 1986, Comput. Vis. Graph. Image Process..

[43]  Thomas S. Huang,et al.  A linear algorithm for motion estimation using straight line correspondences , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.