Simultaneous Localization and Mapping (SLAM) for Autonomous Driving: Concept and Analysis

The Simultaneous Localization and Mapping (SLAM) technique has achieved astonishing progress over the last few decades and has generated considerable interest in the autonomous driving community. With its conceptual roots in navigation and mapping, SLAM outperforms some traditional positioning and localization techniques since it can support more reliable and robust localization, planning, and controlling to meet some key criteria for autonomous driving. In this study the authors first give an overview of the different SLAM implementation approaches and then discuss the applications of SLAM for autonomous driving with respect to different driving scenarios, vehicle system components and the characteristics of the SLAM approaches. The authors then discuss some challenging issues and current solutions when applying SLAM for autonomous driving. Some quantitative quality analysis means to evaluate the characteristics and performance of SLAM systems and to monitor the risk in SLAM estimation are reviewed. In addition, this study describes a real-world road test to demonstrate a multi-sensor-based modernized SLAM procedure for autonomous driving. The numerical results show that a high-precision 3D point cloud map can be generated by the SLAM procedure with the integration of Lidar and GNSS/INS. Online four–five cm accuracy localization solution can be achieved based on this pre-generated map and online Lidar scan matching with a tightly fused inertial system.

[1]  Weifeng Chen,et al.  An Overview on Visual SLAM: From Tradition to Semantic , 2022, Remote. Sens..

[2]  P. From,et al.  Evaluation of RGB-D Multi-Camera Pose Estimation for 3D Reconstruction , 2022, Applied Sciences.

[3]  A. Al-Hamadi,et al.  An Online Semantic Mapping System for Extending and Enhancing Visual SLAM , 2022, Eng. Appl. Artif. Intell..

[4]  Haoyang Zhang.  Deep Learning Applications in Simultaneous Localization and Mapping , 2022, Journal of Physics: Conference Series.

[5]  Qihong Chen,et al.  An Improved Initialization Method for Monocular Visual-Inertial SLAM , 2021, Electronics.

[6]  Andrea Masiero,et al.  The YTU dataset and recurrent neural network based visual-inertial odometry , 2021 .

[7]  Muhammad Sualeh,et al.  Semantics Aware Dynamic SLAM Based on 3D MODT , 2021, Sensors.

[8]  Mingkun Yang,et al.  DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometry , 2021, Neurocomputing.

[9]  Cyrill Stachniss,et al.  Moving Object Segmentation in 3D LiDAR Data: A Learning-Based Approach Exploiting Sequential Data , 2021, IEEE Robotics and Automation Letters.

[10]  Dongbing Gu,et al.  DeepSLAM: A Robust Monocular SLAM System With Unsupervised Deep Learning , 2021, IEEE Transactions on Industrial Electronics.

[11]  Jingnan Liu,et al.  Real-Time HD Map Change Detection for Crowdsourcing Update Based on Mid-to-High-End Sensors , 2021, Sensors.

[12]  Yunzhou Zhang,et al.  Semantic loop closure detection based on graph matching in multi-objects scenes , 2021, J. Vis. Commun. Image Represent..

[13]  Chunyun Fu,et al.  A Review on Map-Merging Methods for Typical Map Types in Multiple-Ground-Robot SLAM Solutions , 2020, Sensors.

[14]  Thiago Oliveira-Santos,et al.  What is the Best Grid-Map for Self-Driving Cars Localization? An Evaluation under Diverse Types of Illumination, Traffic, and Environment , 2020, Expert Syst. Appl..

[15]  Dongsheng Wang,et al.  Multi-camera visual SLAM for off-road navigation , 2020, Robotics Auton. Syst..

[16]  Damien Vivet,et al.  A Review of Visual-LiDAR Fusion based Simultaneous Localization and Mapping , 2020, Sensors.

[17]  Hesheng Wang,et al.  Loop closure detection using supervised and unsupervised deep neural networks for monocular SLAM systems , 2020, Robotics Auton. Syst..

[18]  Zhenhai Gao,et al.  De-Skewing LiDAR Scan for Refinement of Local Mapping , 2020, Sensors.

[19]  Bingqi Zhang,et al.  High Definition Map for Automated Driving: Overview and Analysis , 2020, Journal of Navigation.

[20]  Yahya Zweiri,et al.  Feature-based visual simultaneous localization and mapping: a survey , 2020, SN Applied Sciences.

[21]  Steffen Junginger,et al.  Deep Learning for Visual SLAM in Transportation Robotics: A review , 2019, Transportation Safety and Environment.

[22]  Jaeheung Park,et al.  Parking Line Based SLAM Approach Using AVM/LiDAR Sensor Fusion for Rapid and Accurate Loop Closing and Parking Space Detection , 2019, Sensors.

[23]  Ahmed El-Mowafy,et al.  On detection of observation faults in the observation and position domains for positioning of intelligent transport systems , 2019, Journal of Geodesy.

[24]  Guolai Jiang,et al.  A Simultaneous Localization and Mapping (SLAM) Framework for 2.5D Map Building Based on Low-Cost LiDAR and Vision Fusion , 2019, Applied Sciences.

[25]  Chris Rizos,et al.  On hypothesis testing in RAIM algorithms: generalized likelihood ratio test, solution separation test and a possible alternative , 2019, Measurement Science and Technology.

[26]  Le Chang,et al.  GNSS/INS/LiDAR-SLAM Integrated Navigation System Based on Graph Optimization , 2019, Remote. Sens..

[27]  Yanlei Gu,et al.  Autonomous vehicle self-localization based on abstract map and multi-channel LiDAR in urban area , 2019, IATSS Research.

[28]  Alberto Ferreira de Souza,et al.  Self-Driving Cars: A Survey , 2019, Expert Syst. Appl..

[29]  Myoungho Sunwoo,et al.  Crowd-Sourced Mapping of New Feature Layer for High-Definition Map , 2018, Sensors.

[30]  Dongbing Gu,et al.  Ongoing Evolution of Visual SLAM from Geometry to Deep Learning: Challenges and Opportunities , 2018, Cognitive Computation.

[31]  Javier Ruiz-del-Solar,et al.  Topological Semantic Mapping and Localization in Urban Road Scenarios , 2018, J. Intell. Robotic Syst..

[32]  Myoungho Sunwoo,et al.  Simultaneous Localization and Map Change Update for the High Definition Map-Based Autonomous Driving Car , 2018, Sensors.

[33]  Hua Zhu,et al.  A Review of Visual-Inertial Simultaneous Localization and Mapping from Filtering-Based and Optimization-Based Perspectives , 2018, Robotics.

[34]  Hua Zhu,et al.  Visual-inertial SLAM method based on optical flow in a GPS-denied environment , 2018, Ind. Robot.

[35]  Dongbing Gu,et al.  Semantic Scene Mapping with Spatio-temporal Deep Neural Network for Robotic Applications , 2018, Cognitive Computation.

[36]  Mehrdad Dianati,et al.  A Survey of the State-of-the-Art Localization Techniques and Their Potentials for Autonomous Vehicle Applications , 2018, IEEE Internet of Things Journal.

[37]  Zhen Leng,et al.  Accurate Initial State Estimation in a Monocular Visual–Inertial SLAM System , 2018, Sensors.

[38]  Giuseppe C. Calafiore,et al.  Convex Relaxations for Pose Graph Optimization With Outliers , 2018, IEEE Robotics and Automation Letters.

[39]  Juha Hyyppä,et al.  Graph SLAM correction for single scanner MLS forest data under boreal forest canopy , 2017 .

[40]  Gabriele Costante,et al.  LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation , 2017, IEEE Robotics and Automation Letters.

[41]  S. Glaser,et al.  Simultaneous localization and mapping: A survey of current trends in autonomous driving , 2017 .

[42]  Jun Wang,et al.  Map-Based Localization Method for Autonomous Vehicles Using 3D-LIDAR * , 2017 .

[43]  Andreas Zell,et al.  Multi-camera visual SLAM for autonomous navigation of micro aerial vehicles , 2017, Robotics Auton. Syst..

[44]  Sei Ikeda,et al.  Visual SLAM algorithms: a survey from 2010 to 2016 , 2017, IPSJ Transactions on Computer Vision and Applications.

[45]  Lianyu Zheng,et al.  Real-Time Motion Tracking for Mobile Augmented/Virtual Reality Using Adaptive Visual-Inertial Fusion , 2017, Sensors.

[46]  Luis Miguel Bergasa,et al.  A Multi-Sensorial Simultaneous Localization and Mapping (SLAM) System for Low-Cost Micro Aerial Vehicles in GPS-Denied Environments , 2017, Sensors.

[47]  Viorela Ila,et al.  SLAM++ 1 -A highly efficient and temporally scalable incremental SLAM framework , 2017, Int. J. Robotics Res..

[48]  Sen Wang,et al.  VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem , 2017, AAAI.

[49]  Zeyu Li,et al.  Geometric analysis of reality-based indoor 3D mapping , 2016 .

[50]  Mohammad H. Marhaban,et al.  Review of visual odometry: types, approaches, challenges, and applications , 2016, SpringerPlus.

[51]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[52]  Ruofei Zhong,et al.  Feature-Based Laser Scan Matching and Its Application for Indoor Mapping , 2016, Sensors.

[53]  Xiaolong Hu,et al.  Autonomous Driving in the iCity—HD Maps as a Key Challenge of the Automotive Industry , 2016 .

[54]  Romuald Aufrère,et al.  A general consistent decentralized Simultaneous Localization And Mapping solution , 2015, Robotics Auton. Syst..

[55]  Olaf Kähler,et al.  Very High Frame Rate Volumetric Integration of Depth Images on Mobile Devices , 2015, IEEE Transactions on Visualization and Computer Graphics.

[56]  Christos Katrakazas,et al.  Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions , 2015 .

[57]  Zeng-Guang Hou,et al.  Neural network based FastSLAM for autonomous robots in unknown environments , 2015, Neurocomputing.

[58]  Siavash Hosseinyalamdary,et al.  Tracking 3D Moving Objects Based on GPS/IMU Navigation Solution, Laser Scanner Point Cloud and GIS Data , 2015, ISPRS Int. J. Geo Inf..

[59]  Xiao Feng,et al.  Autonomous navigation based on unscented-FastSLAM using particle swarm optimization for autonomous underwater vehicles , 2015 .

[60]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[61]  Wolfram Burgard,et al.  Survey of Geodetic Mapping Methods: Geodetic Approaches to Mapping and the Relationship to Graph-Based SLAM , 2014, IEEE Robotics & Automation Magazine.

[62]  Marc Pollefeys,et al.  Self-calibration and visual SLAM with a multi-camera system on a micro aerial vehicle , 2014, Auton. Robots.

[63]  Basilio Bona,et al.  A fast and accurate approximation for planar pose graph optimization , 2014, Int. J. Robotics Res..

[64]  Fawzi Nashashibi,et al.  Multivehicle Cooperative Local Mapping: A Methodology Based on Occupancy Grid Map Merging , 2014, IEEE Transactions on Intelligent Transportation Systems.

[65]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[66]  Yasir Latif,et al.  Robust loop closing over time for pose graph SLAM , 2013, Int. J. Robotics Res..

[67]  Cindy Cappelle,et al.  Camera/Laser/GPS Fusion Method for Vehicle Positioning Under Extended NIS-Based Sensor Validation , 2013, IEEE Transactions on Instrumentation and Measurement.

[68]  Roland Chapuis,et al.  Localization and Mapping Using Only a Rotating FMCW Radar Sensor , 2013, Sensors.

[69]  Ling Yang,et al.  Outlier separability analysis with a multiple alternative hypotheses test , 2013, Journal of Geodesy.

[70]  José Ruíz Ascencio,et al.  Visual simultaneous localization and mapping: a survey , 2012, Artificial Intelligence Review.

[71]  Jinling Wang,et al.  New Outlier Separability Test and Its Application in GNSS Positioning , 2012 .

[72]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[73]  Pinhas Ben-Tzvi,et al.  Mapping, localization and motion planning in mobile multi-robotic systems , 2012, Robotica.

[74]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[75]  Frank Dellaert,et al.  iSAM2: Incremental smoothing and mapping using the Bayes tree , 2012, Int. J. Robotics Res..

[76]  Xinzheng Zhang,et al.  Sensor Fusion of Monocular Cameras and Laser Rangefinders for Line-Based Simultaneous Localization and Mapping (SLAM) Tasks in Autonomous Mobile Robots , 2012, Sensors.

[77]  Ian D. Reid,et al.  RSLAM: A System for Large-Scale Mapping in Constant-Time Using Stereo , 2011, International Journal of Computer Vision.

[78]  Stergios I. Roumeliotis,et al.  Observability-based Rules for Designing Consistent EKF SLAM Estimators , 2010, Int. J. Robotics Res..

[79]  Jinling Wang,et al.  Extended Receiver Autonomous Integrity Monitoring (eRAIM) for GNSS/INS Integration , 2010 .

[80]  Gamini Dissanayake,et al.  Iterated D-SLAM map joining: evaluating its performance in terms of consistency, accuracy and efficiency , 2009, Auton. Robots.

[81]  Cyrill Stachniss,et al.  On measuring the accuracy of SLAM algorithms , 2009, Auton. Robots.

[82]  Wolfram Burgard,et al.  Nonlinear Constraint Network Optimization for Efficient Map Learning , 2009, IEEE Transactions on Intelligent Transportation Systems.

[83]  Jinling Wang,et al.  On the Availability of Fault Detection and Exclusion in GNSS Receiver Autonomous Integrity Monitoring , 2009, Journal of Navigation.

[84]  Frank Dellaert,et al.  iSAM: Incremental Smoothing and Mapping , 2008, IEEE Transactions on Robotics.

[85]  Lina María Paz,et al.  Large-Scale 6-DOF SLAM With Stereo-in-Hand , 2008, IEEE Transactions on Robotics.

[86]  Patrick Weber,et al.  OpenStreetMap: User-Generated Street Maps , 2008, IEEE Pervasive Computing.

[87]  Gamini Dissanayake,et al.  Convergence and Consistency Analysis for Extended Kalman Filter Based SLAM , 2007, IEEE Transactions on Robotics.

[88]  Hugh F. Durrant-Whyte,et al.  Simultaneous Localization, Mapping and Moving Object Tracking , 2007, Int. J. Robotics Res..

[89]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[90]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[91]  Tom Drummond,et al.  Scalable Monocular SLAM , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[92]  Tom Duckett,et al.  A multilevel relaxation algorithm for simultaneous localization and mapping , 2005, IEEE Transactions on Robotics.

[93]  Eduardo Mario Nebot,et al.  Solving computational and memory requirements of feature-based simultaneous localization and mapping algorithms , 2003, IEEE Trans. Robotics Autom..

[94]  John J. Leonard,et al.  Robust Mapping and Localization in Indoor Environments Using Sonar Data , 2002, Int. J. Robotics Res..

[95]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[96]  Eduardo Mario Nebot,et al.  Optimization of the simultaneous localization and map-building algorithm for real-time implementation , 2001, IEEE Trans. Robotics Autom..

[97]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[98]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[99]  Hugh F. Durrant-Whyte,et al.  Mobile robot localization by tracking geometric beacons , 1991, IEEE Trans. Robotics Autom..

[100]  Peter Cheeseman,et al.  On the Representation and Estimation of Spatial Uncertainty , 1986 .

[101]  Jonathan M. Roberts,et al.  Field and Service Robotics , 2021, Springer Proceedings in Advanced Robotics.

[102]  Attila Kiss,et al.  MASAT: A fast and robust algorithm for pose-graph initialization , 2020, Pattern Recognit. Lett..

[103]  Zhihong Xi,et al.  Dynamic Scene Semantics SLAM Based on Semantic Segmentation , 2020, IEEE Access.

[104]  Hideo Saito,et al.  Efficient Object-Oriented Semantic Mapping With Object Detector , 2019, IEEE Access.

[105]  Yong Guan,et al.  Manifold Regularization Graph Structure Auto-Encoder to Detect Loop Closure for Visual SLAM , 2019, IEEE Access.

[106]  C. V. Jawahar,et al.  Computer Vision – ACCV 2018 , 2018, Lecture Notes in Computer Science.

[107]  Changle Xiang,et al.  A New Grid Map Construction Method for Autonomous Vehicles , 2018 .

[108]  Tao Zhang,et al.  Unsupervised learning to detect loops using deep neural networks for visual SLAM system , 2017, Auton. Robots.

[109]  Paolo Valigi,et al.  Exploring Representation Learning With CNNs for Frame-to-Frame Ego-Motion Estimation , 2016, IEEE Robotics and Automation Letters.

[110]  Yu Zheng Chong,et al.  Sensor Technologies and Simultaneous Localization and Mapping (SLAM) , 2015 .

[111]  Gerhard Goos,et al.  New Trends in Image Analysis and Processing – ICIAP 2013 , 2013, Lecture Notes in Computer Science.

[112]  Vijay Kumar,et al.  Experimental Robotics, The Eleventh International Symposium, ISER 2008, July 13-16, 2008, Athens, Greece , 2009, ISER.

[113]  José Neira,et al.  Divide and Conquer: EKF SLAM in $O(n)$ , 2008, IEEE Transactions on Robotics.

[114]  Sebastian Thrun,et al.  FastSLAM: An Efficient Solution to the Simultaneous Localization And Mapping Problem with Unknown Data , 2004 .

[115]  Sebastian Thrun,et al.  Multi-robot SLAM with Sparse Extended Information Filers , 2003, ISRR.

[116]  H. Durrant-Whyte,et al.  Simultaneous Localization and Mapping ( SLAM ) : Part II BY TIM BAILEY AND , 2000 .

[117]  John J. Leonard,et al.  A Computationally Efficient Method for Large-Scale Concurrent Mapping and Localization , 2000 .