From Timing Variations to Performance Degradation: Understanding and Mitigating the Impact of Software Execution Timing in SLAM

Timing is an important property for robotic systems that continuously interact with our physical world. Variation in program execution time caused by limited computational resources or system resource contention can lead to significant impact on algorithmic result accuracy. Even though recent work has found Simultaneous Localization And Mapping (SLAM) to be timing-sensitive, little exists in understanding the interactions between the timing variations in SLAM systems and the corresponding degradation. In this paper we conduct a systematic analysis of nine state-of-the-art SLAM systems and dissect the root causes of their degradation. We discovered that timing-induced errors are generated either from delayed execution in certain critical tasks, or from desynchronization in sensor fusion. Based on the insights from our analysis, we propose a solution that combines selective fusion on data in the front end and temporal budget optimization on bundle adjust-ment in the backend to mitigate the impacts of unexpected timing variation adaptively. Experimental results show that our proposed method makes it possible to migrate expensive algorithms to low-cost platforms without laborious tuning, while making the SLAM system robust against the effects of abnormal timing.

[1]  Wenjing Lou,et al.  Decentralized Spectrum Access System: Vision, Challenges, and a Blockchain Solution , 2021, IEEE Wireless Communications.

[2]  Ning Zhang,et al.  Chronos: Timing Interference as a New Attack Vector on Autonomous Cyber-physical Systems , 2021, CCS.

[3]  Qiben Yan,et al.  Security and Privacy in the Emerging Cyber-Physical World: A Survey , 2021, IEEE Communications Surveys & Tutorials.

[4]  Daiki Mori,et al.  Simultaneous Estimation of Vehicle Position and Data Delays using Gaussian Process based Moving Horizon Estimation , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Kyungtae Kang,et al.  R-TOD: Real-Time Object Detector with Minimized End-to-End Delay for Autonomous Driving , 2020, 2020 IEEE Real-Time Systems Symposium (RTSS).

[6]  Ning Zhang,et al.  PCBChain: Lightweight Reconfigurable Blockchain Primitives for Secure IoT Applications , 2020, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[7]  Yipu Zhao,et al.  Good Graph to Optimize: Cost-Effective, Budget-Aware Bundle Adjustment in Visual SLAM , 2020, ArXiv.

[8]  Saurabh Jha,et al.  FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices , 2020, OSDI.

[9]  Carlos Campos,et al.  ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM , 2020, IEEE Transactions on Robotics.

[10]  Geir Hovland,et al.  Evaluation of Perception Latencies in a Human-Robot Collaborative Environment , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Yipu Zhao,et al.  Closed-Loop Benchmarking of Stereo Visual-Inertial SLAM Systems: Understanding the Impact of Drift and Latency on Tracking Accuracy , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Yipu Zhao,et al.  Good Feature Matching: Toward Accurate, Robust VO/VSLAM With Low Latency , 2020, IEEE Transactions on Robotics.

[13]  Qinghua Zheng,et al.  Towards Efficient Fine-Grained Access Control and Trustworthy Data Processing for Remote Monitoring Services in IoT , 2019, IEEE Transactions on Information Forensics and Security.

[14]  Ning Zhang,et al.  A Survey of Distributed Consensus Protocols for Blockchain Networks , 2019, IEEE Communications Surveys & Tutorials.

[15]  Wenjing Lou,et al.  Distributed Consensus Protocols and Algorithms , 2019, Blockchain for Distributed Systems Security.

[16]  Davide Scaramuzza,et al.  How Fast Is Too Fast? The Role of Perception Latency in High-Speed Sense and Avoid , 2019, IEEE Robotics and Automation Letters.

[17]  Luca Carlone,et al.  Attention and Anticipation in Fast Visual-Inertial Navigation , 2019, IEEE Transactions on Robotics.

[18]  Wei Liu,et al.  Modeling Varying Camera-IMU Time Offset in Optimization-Based Visual-Inertial Odometry , 2018, ECCV.

[19]  Shaojie Shen,et al.  Online Temporal Calibration for Monocular Visual-Inertial Systems , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Sushil Jajodia,et al.  Memory Forensic Challenges Under Misused Architectural Features , 2018, IEEE Transactions on Information Forensics and Security.

[21]  Shaojie Shen,et al.  VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator , 2017, IEEE Transactions on Robotics.

[22]  Jonathan P. How,et al.  Two-Stage Focused Inference for Resource-Constrained Minimal Collision Navigation , 2017, IEEE Transactions on Robotics.

[23]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[24]  Roland Siegwart,et al.  The EuRoC micro aerial vehicle datasets , 2016, Int. J. Robotics Res..

[25]  Daniel Cremers,et al.  Direct Sparse Odometry , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Wolfgang Hess,et al.  Real-time loop closure in 2D LIDAR SLAM , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Michael Bosse,et al.  Keyframe-based visual–inertial odometry using nonlinear optimization , 2015, Int. J. Robotics Res..

[28]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Ji Zhang,et al.  LOAM: Lidar Odometry and Mapping in Real-time , 2014, Robotics: Science and Systems.

[30]  Anastasios I. Mourikis,et al.  3-D motion estimation and online temporal calibration for camera-IMU systems , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  Kurt Konolige,et al.  Double window optimisation for constant time visual SLAM , 2011, 2011 International Conference on Computer Vision.

[32]  Paul Newman,et al.  Closing loops without places , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Frank Dellaert,et al.  Covariance recovery from a square root information matrix for data association , 2009, Robotics Auton. Syst..

[34]  Christian Schlegel,et al.  Landmark rating and selection according to localization coverage: Addressing the challenge of lifelong operation of SLAM in service robots , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Ricardo Carelli,et al.  Feature Selection Criteria for Real Time EKF-SLAM Algorithm , 2009 .

[36]  Stergios I. Roumeliotis,et al.  A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[37]  Sen Zhang,et al.  Entropy based feature selection scheme for real time simultaneous localization and map building , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Wolfram Burgard,et al.  Monte Carlo Localization: Efficient Position Estimation for Mobile Robots , 1999, AAAI/IAAI.

[39]  Gaetano Scarano,et al.  Discrete time techniques for time delay estimation , 1993, IEEE Trans. Signal Process..

[40]  Jason Nieh,et al.  Argus: Debugging Performance Issues in Modern Desktop Applications with Annotated Causal Tracing , 2021, USENIX Annual Technical Conference.

[41]  Joshua Fried,et al.  Caladan: Mitigating Interference at Microsecond Timescales , 2020, OSDI.

[42]  Gaurav S. Sukhatme,et al.  A General Framework for Temporal Calibration of Multiple Proprioceptive and Exteroceptive Sensors , 2010, ISER.