Evaluating Egomotion and Structure-from-Motion Approaches Using the TUM RGB-D Benchmark

In this paper, we present the TUM RGB-D benchmark for visual odometry and SLAM evaluation and report on the first use-cases and users of it outside our own group. The benchmark contains a large set of image sequences recorded from a Microsoft Kinect associated with highly accurate and time-synchronized ground truth camera poses from an external motion capture system. The dataset consists in total of 39 sequences that were recorded in different environments and cover a large variety of scenes and camera motions. In this work, we discuss and briefly summarize the evaluation results of the first users from outside our group. Our goal with this analysis is to better understand (1) how other researcher use our dataset to date and (2) how to improve it further in the future.

[1]  Evangelos E. Milios,et al.  Globally Consistent Range Scan Alignment for Environment Mapping , 1997, Auton. Robots.

[2]  Andreas Nüchter,et al.  Automatic appearance-based loop detection from three-dimensional laser data using the normal distributions transform , 2009 .

[3]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Philip R. Osteen,et al.  Online egomotion estimation of RGB-D sensors using spherical harmonics , 2012, IEEE International Conference on Robotics and Automation.

[5]  Achim J. Lilienthal,et al.  Point set registration through minimization of the L2 distance between 3D-NDT models , 2012, 2012 IEEE International Conference on Robotics and Automation.

[6]  Frank Dellaert,et al.  Square Root SAM , 2005, Robotics: Science and Systems.

[7]  Cyrill Stachniss,et al.  On measuring the accuracy of SLAM algorithms , 2009, Auton. Robots.

[8]  Wolfram Burgard,et al.  An evaluation of the RGB-D SLAM system , 2012, 2012 IEEE International Conference on Robotics and Automation.

[9]  Kurt Konolige,et al.  Towards lifelong visual maps , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Roland Siegwart,et al.  Tracking a depth camera: Parameter exploration for fast ICP , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Frank Dellaert,et al.  Incremental smoothing and mapping , 2008 .

[12]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[13]  Luc Van Gool,et al.  From images to 3D models , 2002, CACM.

[14]  Hauke Strasdat,et al.  Scale Drift-Aware Large Scale Monocular SLAM , 2010, Robotics: Science and Systems.

[15]  Winston Churchill,et al.  The New College Vision and Laser Data Set , 2009, Int. J. Robotics Res..

[16]  Kurt Konolige,et al.  Large-Scale Visual Odometry for Rough Terrain , 2007, ISRR.

[17]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2005, Machine Vision and Applications.

[18]  Wolfram Burgard,et al.  Nonlinear Constraint Network Optimization for Efficient Map Learning , 2009, IEEE Transactions on Intelligent Transportation Systems.

[19]  Patrick Rives,et al.  Real-time Quadrifocal Visual Odometry , 2010, Int. J. Robotics Res..

[20]  Alonzo Kelly,et al.  Linearized Error Propagation in Odometry , 2004, Int. J. Robotics Res..

[21]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[23]  Sebastian Thrun,et al.  FastSLAM: a factored solution to the simultaneous localization and mapping problem , 2002, AAAI/IAAI.

[24]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[26]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[27]  Joachim Hertzberg,et al.  Ground truth evaluation of large urban 6D SLAM , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Robert C. Bolles,et al.  Outdoor Mapping and Navigation Using Stereo Vision , 2006, ISER.

[29]  Daniel Cremers,et al.  Real-Time Dense Geometry from a Handheld Camera , 2010, DAGM-Symposium.

[30]  Wolfram Burgard,et al.  Towards a benchmark for RGB-D SLAM evaluation , 2011, RSS 2011.

[31]  Silvio Savarese,et al.  Semantic structure from motion , 2011, CVPR 2011.

[32]  Joachim Hertzberg,et al.  Evolving interface design for robot search tasks: Research Articles , 2007 .

[33]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[34]  Susanne Thierfelder.  Modified Particle Swarm Optimization for a 6DOF Local Pose Estimation Algorithm by Using a RGB-D Camera , 2012 .

[35]  Reinhard Koch,et al.  An Analysis-by-Synthesis Camera Tracking Approach Based on Free-Form Surfaces , 2007, DAGM-Symposium.

[36]  Wolfram Burgard,et al.  Real-time 3D visual SLAM with a hand-held camera , 2011 .

[37]  Wolfram Burgard,et al.  A comparison of SLAM algorithms based on a graph of relations , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Jörg Stückler,et al.  Model Learning and Real-Time Tracking Using Multi-Resolution Surfel Maps , 2012, AAAI.

[39]  Patrick Rives,et al.  Real-time dense RGB-D localisation and mapping , 2011, IEEE International Conference on Robotics and Automation.

[40]  Andrew I. Comport,et al.  Real-time dense appearance-based SLAM for RGB-D sensors , 2011 .

[41]  E. Olson.  Evaluating the Performance of Map Optimization Algorithms , 2009 .

[42]  Joachim Hertzberg,et al.  6D SLAM—3D mapping outdoor environments , 2007, J. Field Robotics.

[43]  Stefano Soatto,et al.  Real-time 3D motion and structure of point features: a front-end system for vision-based control and interaction , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[44]  Andreas Nüchter,et al.  Automatic Appearance-Based Loop Detection from 3 D Laser Data Using the Normal Distributions Transform , 2009 .

[45]  Jörg Stückler,et al.  Robust Real-Time Registration of RGB-D Images using Multi-Resolution Surfel Representations , 2012, ROBOTIK.

[46]  Frank Dellaert,et al.  iSAM: Incremental Smoothing and Mapping , 2008, IEEE Transactions on Robotics.

[47]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[48]  Daniel Cremers,et al.  Real-time visual odometry from dense RGB-D images , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[49]  Wolfram Burgard,et al.  G2o: A general framework for graph optimization , 2011, 2011 IEEE International Conference on Robotics and Automation.