A hierarchical approach for scene segmentation based on 2D motion

This paper deals with the determination of the main components of an outdoor scene from an image sequence observed by a mobile camera. By components, we mean the different depth "layers" of the scene. To segment the scene, we exploit the 2D motion which implicitly contains relative depth information. To achieve this segmentation, 2D affine motion models are considered. Models parameters for each extracted region are estimated from a dense velocity field. Its computation relies on a nonlinear diffusion method which preserves the motion discontinuities and supplies a consistency measure map. These data are used as observations in a hierarchical approach composed of two levels. The local merging step which classifies the pixels into regions, and the global merging step which ensures the consistency of each extracted region. The local merging step is embedded in a Markov random fields formalism, whereas the global merging step is also based on an energy formulation.