Probabilistic Analysis and Extraction of Video Content

In this paper we present a probabilistic framework for mapping low-level visual features into a specific set of semantic descriptors. Specifically, we employ hidden Markov models (HMMs) and Bayesian belief networks (BBNs) at various stages to characterize content domains and extract the relevant semantic information. HMMs are utilized at the shot and sequence levels to model the sequentially-varying structure of video sequences and delineate the video stream in terms of the constituent shots. BBNs, on the-other hand, act on and within each shot, to provide more detailed descriptions of shot content using the physical features of video objects. The semantic content extraction problem is thus addressed at all physical (shot and object) levels, within a consistent representation and processing framework.