SLAM-based 3D Line Reconstruction

Recovering 3D structure of the scene is widely studied in computer vision, which is called structure from motion. Most of the structure from motion approaches using features points to reconstruct 3D point cloud maps to represent scenes. However, there are always many lines in manmade scenes. Line is a higher-level representation of scene structure than point. We contribute a SLAM (simultaneous location and mapping) based method to reconstruct the 3D lines of urban scenes to emphasize the structures. In our approach, keyframe based SLAM system provides keyframes and camera poses, line segments are extracted and tracked in keyframes, the spatial line is determined in two steps: direction then position. The direction is acquired from normal vectors of multiple line projection planes and depth is searched by minimizing the total re-projection error of a spatial line in multiple keyframes. Degenerate and incorrectly tracked cases are detected by rank and principal components analysis. We present accurate 3D line model for urban scene with UAV inspection. The generated 3D lines give a better structural sense than sparse point cloud.

[1]  David J. Kriegman,et al.  Structure and Motion from Line Segments in Multiple Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Zhanyi Hu,et al.  MSLD: A robust descriptor for line matching , 2009, Pattern Recognit..

[3]  Il Hong Suh,et al.  Building a partial 3D line-based map using a monocular SLAM , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Lu Wang,et al.  Wide-baseline image matching using Line Signatures , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[6]  Ian D. Reid,et al.  Real-Time Monocular SLAM with Straight Lines , 2006, BMVC.

[7]  Adrien Bartoli,et al.  Structure-from-motion using lines: Representation, triangulation, and bundle adjustment , 2005, Comput. Vis. Image Underst..

[8]  Horst Bischof,et al.  Semi-Global 3D Line Modeling for Incremental Structure-from-Motion , 2014, BMVC.

[9]  I. Jolliffe.  Principal Component Analysis , 2002 .

[10]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[11]  Reinhard Koch,et al.  An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency , 2013, J. Vis. Commun. Image Represent..

[12]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[13]  Hideo Saito,et al.  Fast Line Description for Line-based SLAM , 2012, BMVC.

[14]  Simon Fuhrmann,et al.  MVE - A Multi-View Reconstruction Environment , 2014, GCH.

[15]  Sukhan Lee,et al.  Simultaneous line matching and epipolar geometry estimation based on the intersection context of coplanar line pairs , 2012, Pattern Recognit. Lett..

[16]  Jana Kosecka,et al.  Detection and matching of rectilinear structures , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Simon Lacroix,et al.  Monocular-vision based SLAM using Line Segments , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[18]  Rafael Grompone von Gioi,et al.  LSD: A Fast Line Segment Detector with a False Detection Control , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Zhanyi Hu,et al.  Robust line matching through line-point invariants , 2012, Pattern Recognit..

[20]  Horst Bischof,et al.  Improving Sparse 3D Models for Man-Made Environments Using Line-Based 3D Reconstruction , 2014, 2014 2nd International Conference on 3D Vision.

[21]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[22]  Manolis I. A. Lourakis,et al.  Matching disparate views of planar surfaces using projective invariants , 2000, Image Vis. Comput..

[23]  Min Chen,et al.  Robust Affine-Invariant Line Matching for High Resolution Remote Sensing Images , 2013 .

[24]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[25]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[26]  Luc Van Gool,et al.  Progressive Prioritized Multi-view Stereo , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Richard Szeliski,et al.  Reconstructing Rome , 2010, Computer.

[28]  Horst Bischof,et al.  Efficient 3D scene abstraction using line segments , 2017, Comput. Vis. Image Underst..

[29]  Bernd Jähne,et al.  Relative Pose Estimation from Straight Lines Using Optical Flow-Based Line Matching and Parallel Line Clustering , 2016, VISIGRAPP.

[30]  Horst Wildenauer,et al.  Structure from Motion with Line Segments Under Relaxed Endpoint Constraints , 2014, 2014 2nd International Conference on 3D Vision.

[31]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[32]  育久 満上,et al.  Bundler: Structure from Motion for Unordered Image Collections , 2011 .

[33]  Horst Bischof,et al.  Line-based 3D Reconstruction of Wiry Objects , 2013 .

[34]  U. Neumann,et al.  Line segment matching and its applications in 3d urban modeling , 2010 .

[35]  Luc Van Gool,et al.  3D from Line Segments in Two Poorly-Textured, Uncalibrated Images , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[36]  Horst Bischof,et al.  Line3D: Efficient 3D Scene Abstraction for the Built Environment , 2015, GCPR.

[37]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[38]  Zhanyi Hu,et al.  Line matching leveraged by point correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Davide Scaramuzza,et al.  The Zurich urban micro aerial vehicle dataset , 2017, Int. J. Robotics Res..