Unsupervised scene analysis: a hidden Markov model approach

This paper presents a new approach to scene analysis, which aims at extracting structured information from a video sequence using directly low-level data. The method models the sequence using a forest of Hidden Markov models (HMMs), which are able to extract two kinds of data, namely, static and dynamic information. The static information results in a segmentation that explains how the chromatic aspect of the static part of the scene evolves. The dynamic information results in the detection of the areas which are more affected by foreground activity. The former is obtained by a spatial clustering of HMMs, resulting in a spatio-temporal segmentation of the video sequence, which is robust to noise and clutter and does not consider the possible moving objects in the scene. The latter is estimated using an entropy-like measure defined on the stationary probability of the Markov chain associated to the HMMs, producing a partition of the scene in activity zones in a consistent and continuous way. The proposed approach constitutes a principled unified probabilistic framework for low level scene analysis and understanding, showing several key features with respect to the state of the art methods, as it extracts information at the lowest possible level (using only pixel gray-level temporal behavior), and is unsupervised in nature. The obtained results on real sequences, both indoor and outdoor, show the efficacy of the proposed approach.