Particle swarm optimization for 3D object tracking in RGB-D images

Abstract 3D object tracking allows augmented reality applications to add virtual content to the real world in a coherent way. With the popularization of RGB-D sensors, new 6 degree-of-freedom tracking techniques that use features extracted from 3D point clouds have been developed, providing more accurate results. This work proposes the use of particle swarm optimization to handle multiple pose hypotheses during top-down tracking from RGB-D images. A fitness function based on 3D point coordinates, color, and normal information was designed, which is able to handle partial object occlusions by applying a threshold to the Euclidean distance between 3D points. The best particle found for a given frame is kept to compose the set of particles of the next swarm and its components are used as an initial pose to define the boundaries of the search space for tracking in the next frame. In addition, we have taken advantage of GPU processing to reduce the running time. Experiments with a publicly available dataset showed that the use of GPU allowed fast object tracking and that the proposed method presents more accurate tracking results in comparison to state-of-the-art optimization-based techniques, especially in situations where objects are partially occluded.