Direct Disparity Space: Robust and Real-time Visual Odometry

We present a direct visual odometry formulation using a warping function in disparity space. In disparity space measurement noise is well-modeled by a Gaussian distribution, in contrast to the heteroscedastic noise in 3D space. In addition, the Jacobian of the warp separates the rotation and translation terms, enabling motion to be estimated from all image points even those located at infinity. Furthermore, we show that direct camera tracking can obtain accurate and robust performance using only a fraction of the image pixels through a simple and efficient pixel selection strategy. Our approach allows faster than real-time computation on a single CPU core with unoptimized code. As our approach does not rely on feature extraction, the selected pixels over successive frames are often unique. Hence, triangulating the selected pixels to the world frame produces an accurate and dense 3D reconstruction with minimal computational cost making it appealing to robotics and embedded applications. We evaluate the performance of our approach against state-of-the-art methods on a range of urban and indoor datasets. We show that our algorithm produces competitive performance, requires no specialized tuning, and continues to produce competitive results even when run with low resolution images where other techniques fail to operate.

[1]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[2]  Adrien Bartoli Groupwise Geometric and Photometric Direct Image Registration , 2006, BMVC.

[3]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision , 2004 .

[4]  Robert M. Haralick,et al.  Review and analysis of solutions of the three point perspective pose estimation problem , 1994, International Journal of Computer Vision.

[5]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[6]  S. Negahdaripour,et al.  Relaxing the Brightness Constancy Assumption in Computing Optical Flow , 1987 .

[7]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[9]  Daniel Cremers,et al.  Real-time visual odometry from dense RGB-D images , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[10]  Andrew J. Davison,et al.  Real-Time Camera Tracking: When is High Frame-Rate Best? , 2012, ECCV.

[11]  Zheng Fang,et al.  Experimental study of odometry estimation methods using RGB-D cameras , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[13]  Andrew Howard,et al.  Real-time stereo visual odometry for autonomous ground vehicles , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Daniel Cremers,et al.  Efficient Dense Scene Flow from Sparse or Dense Stereo Data , 2008, ECCV.

[15]  David R. Musser,et al.  Introspective Sorting and Selection Algorithms , 1997, Softw. Pract. Exp..

[16]  Larry H. Matthies,et al.  Error modeling in stereo navigation , 1986, IEEE J. Robotics Autom..

[17]  Joni-Kristian Kämäräinen,et al.  Live RGB-D camera tracking for television production studios , 2014, J. Vis. Commun. Image Represent..

[18]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[19]  Daniel Cremers,et al.  Large-Scale Multi-resolution Surface Reconstruction from RGB-D Sequences , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  J. Tukey,et al.  The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data , 1974 .

[21]  Hauke Strasdat,et al.  Visual SLAM: Why filter? , 2012, Image Vis. Comput..

[22]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[23]  Adrien Bartoli,et al.  Groupwise Geometric and Photometric Direct Image Registration , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Fukui Kazuhiro,et al.  Realistic CG Stereo Image Dataset With Ground Truth Disparity Maps , 2012 .

[25]  Patrick Rives,et al.  A spherical robot-centered representation for urban navigation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Trevor Darrell,et al.  Motion Estimation from Disparity Images , 2001, ICCV.

[28]  Frank Dellaert,et al.  Flow separation for fast and robust stereo odometry , 2009, 2009 IEEE International Conference on Robotics and Automation.

[29]  Hernán Badino,et al.  Integrating Disparity Images by Incorporating Disparity Rate , 2008, RobVis.

[30]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[31]  Alan M. McIvor,et al.  Evaluation of subpixel feature localization methods for precision measurement , 1994, Other Conferences.

[32]  R. Wolke,et al.  Iteratively Reweighted Least Squares: Algorithms, Convergence Analysis, and Numerical Comparisons , 1988 .

[33]  Tommi Tykkala,et al.  Direct Iterative Closest Point for real-time visual odometry , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[34]  Brett Browning,et al.  Evaluating Pose Estimation Methods for Stereo Visual Odometry on Robots , 2010 .

[35]  I. Du,et al.  Direct Methods , 1998 .

[36]  Laura Sevilla-Lara,et al.  Distribution fields for tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Georgios D. Evangelidis,et al.  Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[39]  Larry H. Matthies,et al.  Kalman filter-based algorithms for estimating depth from image sequences , 1989, International Journal of Computer Vision.

[40]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[41]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[45]  Daniel Cremers,et al.  CopyMe3D: Scanning and Printing Persons in 3D , 2013, GCPR.

[46]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[47]  Atsuto Maki,et al.  Towards a simulation driven stereo vision system , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[48]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[49]  Frank Dellaert,et al.  Fast Image-Based Tracking by Selective Pixel Integration , 2011 .

[50]  Clark F. Olson,et al.  Stereo ego-motion improvements for robust rover navigation , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[51]  John J. Leonard,et al.  Robust real-time visual odometry for dense RGB-D mapping , 2013, 2013 IEEE International Conference on Robotics and Automation.

[52]  Rupert Brooks,et al.  Efficient and reliable methods for direct parameterized image registration , 2008 .

[53]  Jose Luis Blanco,et al.  A tutorial on SE(3) transformation parameterizations and on-manifold optimization , 2012 .

[54]  Alois Knoll,et al.  Efficient compositional approaches for real-time robust direct visual odometry from RGB-D data , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[55]  Patrick Rives,et al.  Real-time Quadrifocal Visual Odometry , 2010, Int. J. Robotics Res..

[56]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[57]  Ian D. Reid,et al.  A Constant-Time Efficient Stereo SLAM System , 2009, BMVC.

[58]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[59]  Andrew I. Comport,et al.  Real-time dense appearance-based SLAM for RGB-D sensors , 2011 .

[60]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[61]  Daniel Cremers,et al.  Submap-Based Bundle Adjustment for 3D Reconstruction from RGB-D Data , 2014, GCPR.

[62]  Vincent Lepetit,et al.  Robust 3D Tracking with Descriptor Fields , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Andrew Zisserman,et al.  Feature Based Methods for Structure and Motion Estimation , 1999, Workshop on Vision Algorithms.

[64]  Reinhard Klette,et al.  Generalised residual images' effect on illumination artifact removal for correspondence algorithms , 2011, Pattern Recognit..

[65]  Frank Dellaert,et al.  Jacobian images of super-resolved texture maps for model-based motion estimation and tracking , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[66]  Patrick Rives,et al.  An Efficient Direct Approach to Visual SLAM , 2008, IEEE Transactions on Robotics.

[67]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[68]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[69]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[70]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..