Learning multi-planar scene models in multi-camera videos

Many man-made environments are constructed with multiple levels where people walk, joined by stairs, ramps and overpasses. This study proposes a novel method to learn the geometry of a scene containing more than a single ground plane by tracking pedestrians and combining information from multiple views. The method estimates a scene model with multiple planes by measuring the variation of pedestrian heights across each camera's field of view. It segments the image into separate plane regions, estimating the relative depth and altitude for each image pixel, thus building a three-dimensional reconstruction of the scene. By estimating the multiple planes, the method enables tracking algorithms to follow objects (pedestrians and/or vehicles) that are moving on different ground planes in the scene. The authors also introduce what they believe is the first public dataset with pedestrian traffic on multiple planes to encourage other researchers to compare their work in this field.