Line Flow based SLAM

We propose a method of visual SLAM by predicting and updating line flows that represent sequential 2D projections of 3D line segments. While indirect SLAM methods using points and line segments have achieved excellent results, they still face problems in challenging scenarios such as occlusions, image blur, and repetitive textures. To deal with these problems, we leverage line flows which encode the coherence of 2D and 3D line segments in spatial and temporal domains as the sequence of all the 2D line segments corresponding to a specific 3D line segment. Thanks to the line flow representation, the corresponding 2D line segment in a new frame can be predicted based on 2D and 3D line segment motions. We create, update, merge, and discard line flows on-the-fly. We model our Line Flow-based SLAM (LF-SLAM) using a Bayesian network. We perform short-term optimization in front-end, and long-term optimization in back-end. The constraints introduced in line flows improve the performance of our LF-SLAM. Extensive experimental results demonstrate that our method achieves better performance than state-of-the-art direct and indirect SLAM approaches. Specifically, it obtains good localization and mapping results in challenging scenes with occlusions, image blur, and repetitive textures.

[1]  Hans-Peter Seidel,et al.  Exploiting global connectivity constraints for reconstruction of 3D line segments from images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Zichen Zhang,et al.  Incremental 3D Line Segment Extraction from Semi-dense SLAM , 2017, 2018 24th International Conference on Pattern Recognition (ICPR).

[3]  Javier Gonzalez-Jimenez,et al.  PL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments , 2017, IEEE Transactions on Robotics.

[4]  Horst Bischof,et al.  Efficient 3D scene abstraction using line segments , 2017, Comput. Vis. Image Underst..

[5]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[6]  Danping Zou,et al.  StructSLAM: Visual SLAM With Building Structure Lines , 2015, IEEE Transactions on Vehicular Technology.

[7]  Zhanyi Hu,et al.  MSLD: A robust descriptor for line matching , 2009, Pattern Recognit..

[8]  Jian Yao,et al.  A Monocular SLAM System Leveraging Structural Regularity in Manhattan World , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Adrien Bartoli,et al.  Structure-from-motion using lines: Representation, triangulation, and bundle adjustment , 2005, Comput. Vis. Image Underst..

[10]  Frank Dellaert,et al.  iSAM: Incremental Smoothing and Mapping , 2008, IEEE Transactions on Robotics.

[11]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[13]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[14]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Zhanyi Hu,et al.  Robust line matching through line-point invariants , 2012, Pattern Recognit..

[16]  Juan D. Tardós,et al.  Fast relocalisation and loop closing in keyframe-based SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Yipu Zhao,et al.  Good Line Cutting: Towards Accurate Pose Tracking of Line-Assisted VO/VSLAM , 2018, ECCV.

[18]  Dinesh Atchuthan,et al.  A micro Lie theory for state estimation in robotics , 2018, ArXiv.

[19]  Reinhard Koch,et al.  Structure and motion from line correspondences: Representation, projection, initialization and sparse bundle adjustment , 2014, J. Vis. Commun. Image Represent..

[20]  Alan Yuille,et al.  A Novel Linelet-Based Representation for Line Segment Detection , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Zhiheng Wang,et al.  HLD: A robust descriptor for line matching , 2009, 2009 11th IEEE International Conference on Computer-Aided Design and Computer Graphics.

[22]  Teresa A. Vidal-Calleja,et al.  Impact of Landmark Parametrization on Monocular EKF-SLAM with Points and Lines , 2011, International Journal of Computer Vision.

[23]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[24]  Yong Liu,et al.  Robust visual SLAM with point and line features , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[26]  James H. Elder,et al.  MCMLSD: A Dynamic Programming Approach to Line Segment Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[28]  Andrew W. Fitzgibbon,et al.  Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Reinhard Koch,et al.  An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency , 2013, J. Vis. Commun. Image Represent..

[30]  Il Hong Suh,et al.  Building a 3-D Line-Based Map Using Stereo SLAM , 2015, IEEE Transactions on Robotics.

[31]  José M. Cañas,et al.  LineSLAM: Visual Real Time Localization Using Lines and UKF , 2013, ROBOT.

[32]  Roland Siegwart,et al.  The EuRoC micro aerial vehicle datasets , 2016, Int. J. Robotics Res..

[33]  Horst Wildenauer,et al.  Structure from Motion with Line Segments Under Relaxed Endpoint Constraints , 2014, 2014 2nd International Conference on 3D Vision.

[34]  Rafael Grompone von Gioi,et al.  LSD: a Line Segment Detector , 2012, Image Process. Line.

[35]  Daniel Cremers,et al.  Direct Sparse Odometry , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Simon Lacroix,et al.  Line-based monocular graph SLAM , 2017, 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[37]  Kyungdon Joo,et al.  Globally Optimal Inlier Set Maximization for Atlanta Frame Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Cuneyt Akinlar,et al.  EDLines: A real-time line segment detector with a false detection control , 2011, Pattern Recognit. Lett..

[39]  Rafael Grompone von Gioi,et al.  LSD: A Fast Line Segment Detector with a False Detection Control , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Frank Dellaert,et al.  Incremental smoothing and mapping , 2008 .

[41]  Yan Lu,et al.  Robust RGB-D Odometry Using Point and Line Features , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[42]  Pascal Monasse,et al.  Multiscale line segment detector for robust and accurate SfM , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[43]  Shenghua Gao,et al.  PPGNet: Learning Point-Pair Graph for Line Segment Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Nicholas Ayache,et al.  Efficient registration of stereo images by matching graph descriptions of edge segments , 1987, International Journal of Computer Vision.

[45]  Francesc Moreno-Noguer,et al.  PL-SLAM: Real-time monocular visual SLAM with points and lines , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[46]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[47]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[48]  Gui-Song Xia,et al.  Learning Attraction Field Representation for Robust Line Segment Detection , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).