Object behavior-based indexing framework for video

In this paper we present a new method for indexing video data based on the descriptions of object behaviors within a scene. The behaviors of objects over time is the most important feature in a scene and make up the semantic contents of the scene. The indices generated based on these temporal features are complementary to still image features for achieving fast and efficient search and retrieval of the video contents. To provide the conceptual framework for indexing based on the object behaviors we propose an object Behavior Description Scheme (BDS) for video data. The Behavior Description Scheme is based on spatio-temporal features and the associated object behavior descriptions are directly extractable from the video data itself (and support data) facilitating automatic indexing.