An evaluation of real-time RGB-D visual odometry algorithms on mobile devices

Abstract We present an evaluation and a comparison of different visual odometry algorithms selected to be tested on a mobile device equipped with a RGB-D camera. The most promising algorithms from the literature are tested on different mobile devices, some equipped with the Structure Sensor. We evaluate the accuracy of each selected algorithm as well as the memory and CPU consumption, and we show that even without specific optimization, some of them can provide a reliable measure of the device motion in real time.

[1]  Wolfram Burgard,et al.  An evaluation of the RGB-D SLAM system , 2012, 2012 IEEE International Conference on Robotics and Automation.

[2]  Jörg Stückler,et al.  Multi-resolution surfel maps for efficient dense 3D modeling and tracking , 2014, J. Vis. Commun. Image Represent..

[3]  Torsten Sattler,et al.  3D Modeling on the Go: Interactive 3D Reconstruction of Large-Scale Scenes on Mobile Devices , 2015, 2015 International Conference on 3D Vision.

[4]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[5]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[6]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[7]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[8]  Marc Pollefeys,et al.  Live Metric 3D Reconstruction on Mobile Phones , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  John J. Leonard,et al.  Robust real-time visual odometry for dense RGB-D mapping , 2013, 2013 IEEE International Conference on Robotics and Automation.

[10]  Olaf Kähler,et al.  Real-Time 3D Tracking and Reconstruction on Mobile Phones , 2015, IEEE Transactions on Visualization and Computer Graphics.

[11]  Todor Stoyanov,et al.  Real time registration of RGB-D data using local visual features and 3D-NDT registration , 2012, ICRA 2012.

[12]  Zheng Fang,et al.  Experimental Evaluation of RGB-D Visual Odometry Methods , 2015 .

[13]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[14]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[15]  A. Bab-Hadiashar,et al.  An Overview to Visual Odometry and Visual SLAM: Applications to Mobile Robotics , 2015 .

[16]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Graeme Jones,et al.  Accurate and Computationally-inexpensive Recovery of Ego-Motion using Optical Flow and Range Flow with Extended Temporal Support , 2013, BMVC.

[18]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[19]  Andrew J. Davison,et al.  A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Roland Siegwart,et al.  A Review of Point Cloud Registration Algorithms for Mobile Robotics , 2015, Found. Trends Robotics.

[21]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[22]  Pushmeet Kohli,et al.  MobileFusion: Real-Time Volumetric Surface Reconstruction and Dense Tracking on Mobile Phones , 2015, IEEE Transactions on Visualization and Computer Graphics.

[23]  Siddhartha S. Srinivasa,et al.  Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields , 2015, Robotics: Science and Systems.

[24]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[25]  Liqiang Feng,et al.  Navigating Mobile Robots: Systems and Techniques , 1996 .

[26]  Daniel Cremers,et al.  Real-time visual odometry from dense RGB-D images , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[27]  Karel Driesen,et al.  The direct cost of virtual function calls in C++ , 1996, OOPSLA '96.

[28]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[29]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[30]  Mordechai Ben-Ari,et al.  Principles of concurrent and distributed programming , 2005, PHI Series in computer science.

[31]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[32]  Luigi di Stefano,et al.  Fusion of Inertial and Visual Measurements for RGB-D SLAM on Mobile Devices , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[33]  Daoqi Yang,et al.  C++ and Object-Oriented Numeric Computing for Scientists and Engineers , 2000, Springer New York.

[34]  Ji Zhang,et al.  A real-time method for depth enhanced visual odometry , 2017, Auton. Robots.

[35]  Klaus Schöffmann,et al.  OpenCV Performance Measurements on Mobile Devices , 2014, ICMR.

[36]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Shahram Izadi,et al.  Modeling Kinect Sensor Noise for Improved 3D Reconstruction and Tracking , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[38]  Islam S. M. Khalil,et al.  Precise Localization and Control of Catalytic Janus Micromotors Using Weak Magnetic Fields , 2015 .

[39]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[40]  Daniel Cremers,et al.  Semi-dense visual odometry for AR on a smartphone , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[41]  Marc Pollefeys,et al.  Turning Mobile Phones into 3D Scanners , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[43]  Jizhong Xiao,et al.  Fast visual odometry and mapping from RGB-D data , 2013, 2013 IEEE International Conference on Robotics and Automation.

[44]  Olaf Kähler,et al.  Simultaneous 3D tracking and reconstruction on a mobile phone , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[45]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[46]  Lukas Furst.  Concurrent Programming Algorithms Principles And Foundations , 2016 .

[47]  Ji Zhang,et al.  Real-time depth enhanced monocular odometry , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[48]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[50]  José García Rodríguez,et al.  A Comparative Study of Registration Methods for RGB-D Video of Static Scenes , 2014, Sensors.

[51]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[52]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[53]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.