Real-Time Cloud Visual Simultaneous Localization and Mapping for Indoor Service Robots

Unlike traditional industrial robots, indoor service robots are usually required to possess high intelligence, such as the skills of flexible moving, precise spacial perceiving. And high intelligence is always accompanied by consuming complicated and expensive computation resources. One solution for indoor service robots is centralization of expensive computation resource so that it is possible to design a low cost client with a high-intelligence brain. However, as a fundamental intelligence function for mobile indoor robots, if a real-time visual Simultaneously Localization and Mapping (vSLAM) system is split into client and brain, it will be confronted with new challenges, such as the barrier of instant data sharing and performance degradation brought by network delay inbetween. To solve the problem, we focus on a framework and approach of cloud-based visual SLAM in this paper, and provide an efficient solution to offload the expensive computation and reduce the cost of robot clients. The integrated system is distributed in a 3-level Cloud with light-weight tracking, high precision dense mapping, and map sharing. Based on recent excellent algorithms, our system is able to run a real-time sparse tracking on the client, and a real-time dense mapping on the cloud server, which outputs an explicit 3D dense map. Only keyframes are sent to the local cloud center to reduce the network bandwidth requirement. Dense geometric pose estimation besides feature-based methods is computed to make the system resistant to feature-less indoor scenes. The camera poses associated with keyframes are optimized on the local computing cloud center, and are sent back to the client to decrease the trajectory drift. We evaluate the system on the Technical University of Munich (TUM) datasets, Imperial College London and National University of Ireland Maynooth (ICL-NUIM) datasets, and the real data captured by our robot in terms of visual odometry on the client side and dense maps generated on the server cloud. Qualitative and quantitative experiments show our cloud visual SLAM system is able to bear the network delay in Local Area Network (LAN), and it is an efficient vSLAM solution for indoor service robots with high intelligence from a centric brain.

[1]  Paul Newman,et al.  Closing loops without places , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Long Wang,et al.  Cooperative Multi-Robot Monocular-SLAM Using Salient Landmarks , 2009, 2009 International Asia Conference on Informatics in Control, Automation and Robotics.

[3]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[4]  David W. Murray,et al.  Simultaneous Localization and Map-Building Using Active Vision , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  John J. Leonard,et al.  Robust real-time visual odometry for dense RGB-D mapping , 2013, 2013 IEEE International Conference on Robotics and Automation.

[6]  Javier Civera,et al.  RGBDTAM: A cost-effective and accurate RGB-D tracking and mapping system , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Lu Fang,et al.  Beyond SIFT using binary features in Loop Closure Detection , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Ian D. Reid,et al.  Real-Time Monocular SLAM with Straight Lines , 2006, BMVC.

[9]  Lino Marques,et al.  A Distributed and Multithreaded SLAM Architecture for Robotic Clusters and Wireless Sensor Networks , 2015, Advances in Social Media Analysis.

[10]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[11]  Simon Lacroix,et al.  Vision-Based SLAM: Stereo and Monocular Approaches , 2007, International Journal of Computer Vision.

[12]  Wolfram Burgard,et al.  A Tutorial on Graph-Based SLAM , 2010, IEEE Intelligent Transportation Systems Magazine.

[13]  Paolo Dario,et al.  Enabling Global Robot Navigation Based on a Cloud Robotics Approach , 2016, Int. J. Soc. Robotics.

[14]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Senthil Yogamani,et al.  Visual SLAM for Automated Driving: Exploring the Applications of Deep Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[17]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[18]  Xiaojun Wu,et al.  DAvinCi: A cloud computing framework for service robots , 2010, 2010 IEEE International Conference on Robotics and Automation.

[19]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[20]  Ian D. Reid,et al.  Locally Planar Patch Features for Real-Time Structure from Motion , 2004, BMVC.

[21]  Andrew J. Davison,et al.  Real-time simultaneous localisation and mapping with a single camera , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  Tim Weyrich,et al.  Real-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion , 2013, 2013 International Conference on 3D Vision.

[23]  Hao Li,et al.  Collaborative Visual SLAM Framework for a Multi-UAVs System Based on Mutually Loop Closing , 2019, WISATS.

[24]  Javier Civera,et al.  C2TAM: A Cloud framework for cooperative tracking and mapping , 2014, Robotics Auton. Syst..

[25]  Jörg Stückler,et al.  Dense Continuous-Time Tracking and Mapping with Rolling Shutter RGB-D Cameras , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[27]  Shaowu Yang,et al.  Deep Learning Based Semantic Labelling of 3D Point Cloud in Visual SLAM , 2018 .

[28]  Yong Liu,et al.  Robust visual SLAM with point and line features , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Jianwei Zhang,et al.  A cloud computing approach to complex robot vision tasks using smart camera systems , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Patrick Benavidez,et al.  Cloud-based realtime robotic Visual SLAM , 2015, 2015 Annual IEEE Systems Conference (SysCon) Proceedings.

[31]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[32]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Danping Zou,et al.  CoSLAM: Collaborative Visual SLAM in Dynamic Environments , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Stefan Leutenegger,et al.  ElasticFusion: Dense SLAM Without A Pose Graph , 2015, Robotics: Science and Systems.

[35]  Huaimin Wang,et al.  Cloud-Based Framework for Scalable and Real-Time Multi-Robot SLAM , 2018, 2018 IEEE International Conference on Web Services (ICWS).

[36]  Pieter Abbeel,et al.  Image Object Label 3 D CAD Model Candidate Grasps Google Object Recognition Engine Google Cloud Storage Select Feasible Grasp with Highest Success Probability Pose EstimationCamera Robots Cloud 3 D Sensor , 2014 .

[37]  Raffaello D'Andrea,et al.  Rapyuta: The RoboEarth Cloud Engine , 2013, 2013 IEEE International Conference on Robotics and Automation.

[38]  Horst Bischof,et al.  Dense reconstruction on-the-fly , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Ji Zhang,et al.  Visual-lidar odometry and mapping: low-drift, robust, and fast , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[40]  Andrew J. Davison,et al.  A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Mayank Singh,et al.  Cloud-Based Collaborative 3D Mapping in Real-Time With Low-Cost Robots , 2015, IEEE Transactions on Automation Science and Engineering.

[42]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Alfredo Favenza,et al.  A Cloud-Based Approach to GNSS Augmentation for Navigation Services , 2014, 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing.

[44]  Dorian Gálvez-López,et al.  Bags of Binary Words for Fast Place Recognition in Image Sequences , 2012, IEEE Transactions on Robotics.

[45]  Torsten Sattler,et al.  Direct visual odometry for a fisheye-stereo camera , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[46]  Wolfram Burgard,et al.  G2o: A general framework for graph optimization , 2011, 2011 IEEE International Conference on Robotics and Automation.

[47]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[48]  Reinhard Klein,et al.  Eurographics Symposium on Point-based Graphics (2006) Octree-based Point-cloud Compression , 2022 .