Dense Tracking with Range Cameras Using Key Frames

We present a low cost localization system that exploits dense image information to continuously track the position of a range camera in 6DOF. This work has two main contributions: First, the localization of the camera is performed with respect to a set of keyframes selected according to a spatial criteria producing a less populated and more uniform distribution of keyframes in space. This allows us to avoid the computational overload caused by having to estimate a depthmap at the frame rate of the camera as it is common in other dense sequential methods. Second, we propose a two-stage approach to compute the current location of the camera with respect to its closest keyframe. During the first stage, our system calculates an initial relative pose estimate from a sparse set of 3D to 2D point correspondences. This estimate is then refined during the second stage using a dense image alignment. The refinement step is stated as a Non Linear Least Squares (NLQs) optimisation embedded in a coarse to fine approach that minimizes the photo-consistency error between the current image and a warped version of the image associated to the closest keyframe and its depth map.