Catch the Shadow: Person Tracking Under Occlusion with a Single RGB-D Camera

Locomotion in physical space is one of the most natural forms of interaction in applications such as virtual reality systems. Although there are many algorithms to track walking people, existing methods mostly fail to cope with occluded bodies in the setting of multi-person tracking with one camera. This paper proposes a method to overcome this challenge by fusing skeletal with shadow data, both of which are captured by a single RGB-D camera. Skeletal tracking provides the positions of people that can be captured directly, while their shadows are used to track them when they are no longer visible. Our experiments confirm that this method can efficiently handle full occlusions. It thus has substantial value in resolving the occlusion problem in multi-person tracking, even with other kinds of cameras.