Cascaded particle filter for real-time tracking using RGB-D sensor

This paper presents a real-time human body tracking system based on cascaded particle filter using Microsoft Kinect. Our tracking is performed in two layers: RGB image-based 2D region body tracking and 1D depth image body extremities tracking. These two layers are combined to represent the basis for full 3D human body tracking by obtaining a probable human body bounding box in 2D while 3D position is obtained by incorporating available depth information. As the first layer, we utilized motion history for tracking a walking person. One particle filter is used to track the 3D bounding box of the person in synchronization with RGB and depth video stream. For this layer, we utilized the expected motion constraints for enhancing the distribution of particle in importance sampling stage. For the second layer within the bounding box, we use depth information to track the extremities. State of the points of extremities is defined and another particle filter is implemented which utilizes a measure using the notion of spin image. The proposed framework can be used to evaluate and compare estimates at any given instances. For example, the initial estimates associated with the tracking of the whole body can be used as a coarse measure for the tracking of the local extremities in case of self-occlusions.