Realization of CUDA-based real-time multi-camera visual SLAM in embedded systems

The real-time capability of multi-camera visual simultaneous localization and mapping (SLAM) in embedded systems is vital for robotic autonomous navigation. However, owing to the incredibly time-consuming feature extraction, multi-camera visual SLAM has high computational complexity and is difficult to run in real-time in embedded systems. This study proposes a central processing unit and graphics processing unit (CPU–GPU) combination acceleration strategy for multi-camera visual SLAM to solve the computational complexity problem, improve computational efficiency, and realize real-time running in embedded systems. First, the GPU-based feature extraction acceleration algorithm is introduced for multi-camera visual SLAM to accelerate the time-consuming feature extraction by using compute unified device architecture to parallelize feature extraction algorithm. Then, a CPU-based multi-threading pipelining method that conducts image reading, feature extraction, and tracking concurrently is proposed to improve the computational efficiency of multi-camera visual SLAM by solving the load imbalance problem caused by GPU use and improving the use of computing resources. Extensive experiment results demonstrate that the improved multi-camera visual SLAM has a speed of 15 frames per second in embedded systems and meets the real-time requirement. Moreover, the improved multi-camera visual SLAM is three times faster than the original CPU-based method. Our open-source code can be found online: https://github.com/CASHIPS-ComputerVision .

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Elsayed E. Hemayed,et al.  Speeding up spatiotemporal feature extraction using GPU , 2018, Journal of Real-Time Image Processing.

[3]  Mehdi Rezaeian,et al.  Towards Affordable Computing: SiftCU a Simple but Elegant GPU-based Implementation of SIFT , 2014 .

[4]  Xiaohua Shi,et al.  Computing OpenSURF on OpenCL and General Purpose GPU , 2013 .

[5]  Jie Cheng,et al.  CUDA by Example: An Introduction to General-Purpose GPU Programming , 2010, Scalable Comput. Pract. Exp..

[6]  Juan D. Tardós,et al.  Fast relocalisation and loop closing in keyframe-based SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Wolfgang Förstner,et al.  Real-Time Accurate Geo-Localization of a MAV with Omnidirectional Visual Odometry and GPS , 2014, ECCV Workshops.

[8]  Ji Xiang,et al.  Parallelization and Optimization of SIFT on GPU Using CUDA , 2013, 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing.

[9]  Abdelhafid Elouardi,et al.  Graph-based SLAM embedded implementation on low-cost architectures: A practical approach , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Mohamed Abouzahir,et al.  Large-scale monocular FastSLAM2.0 acceleration on an embedded heterogeneous architecture , 2016, EURASIP J. Adv. Signal Process..

[11]  Stefan Hinz,et al.  MultiCol-SLAM - A Modular Real-Time Multi-Camera SLAM System , 2016, ArXiv.

[12]  Edgar J. Lobaton,et al.  Building an Integrated Mobile Robotic System for Real-Time Applications in Construction , 2018, Proceedings of the 35th International Symposium on Automation and Robotics in Construction (ISARC).

[13]  Frank Dellaert,et al.  Probabilistic structure matching for visual SLAM with a multi-camera rig , 2010, Comput. Vis. Image Underst..

[14]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[15]  Steven Lake Waslander,et al.  Taming the North: Multi-camera Parallel Tracking and Mapping in Snow-Laden Environments , 2015, FSR.

[16]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[17]  Quan Pan,et al.  Pose estimation for multi-camera systems , 2017, 2017 IEEE International Conference on Unmanned Systems (ICUS).

[18]  Tian Lan,et al.  Research and implementation of 3D SLAM algorithm based on Kinect depth sensor , 2016, 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[19]  Robert Wittmann,et al.  Fast object approximation for real-time 3D obstacle avoidance with biped robots , 2016, 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[20]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[21]  Cyrill Stachniss,et al.  Fast and effective online pose estimation and mapping for UAVs , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Yi Li,et al.  Parallelizing image feature extraction algorithms on multi-core platforms , 2016, J. Parallel Distributed Comput..

[23]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[24]  Elcio Hideiti Shiguemori,et al.  Embedded System for Visual Odometry and Localization of Moving Objects in Images Acquired by Unmanned Aerial Vehicles , 2013, 2013 III Brazilian Symposium on Computing Systems Engineering.

[25]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[26]  Nan Zhang,et al.  Computing Optimised Parallel Speeded-Up Robust Features (P-SURF) on Multi-Core Processors , 2010, International Journal of Parallel Programming.

[27]  Adam Harmat,et al.  Multi-Camera Tracking and Mapping for Unmanned Aerial Vehicles in Unstructured Environments , 2014, Journal of Intelligent & Robotic Systems.

[28]  Chunbo Xiu,et al.  Improved RGB-D vision SLAM algorithm for mobile robot , 2017, 2017 29th Chinese Control And Decision Conference (CCDC).

[29]  Luis Enrique Sucar,et al.  A Fast Algorithm for Robot Localization Using Multiple Sensing Units , 2018, MCPR.

[30]  Qi Zhang,et al.  SIFT implementation and optimization for multi-core systems , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[31]  Sathish S. Vadhiyar,et al.  A real-time implementation of SIFT using GPU , 2018, Journal of Real-Time Image Processing.

[32]  Xiyang Zhi,et al.  Realization of CUDA-based real-time registration and target localization for high-resolution video images , 2016, Journal of Real-Time Image Processing.

[33]  Sven Behnke,et al.  Efficient multi-camera visual-inertial SLAM for micro aerial vehicles , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).