STRG-Index: spatio-temporal region graph indexing for large video databases

In this paper, we propose new graph-based data structure and indexing to organize and retrieve video data. Several researches have shown that a graph can be a better candidate for modeling semantically rich and complicated multimedia data. However, there are few methods that consider the temporal feature of video data, which is a distinguishable and representative characteristic when compared with other multimedia (i.e., images). In order to consider the temporal feature effectively and efficiently, we propose a new graph-based data structure called Spatio-Temporal Region Graph (STRG). Unlike existing graph-based data structures which provide only spatial features, the proposed STRG further provides temporal features, which represent temporal relationships among spatial objects. The STRG is decomposed into its subgraphs in which redundant subgraphs are eliminated to reduce the index size and search time, because the computational complexity of graph matching (subgraph isomorphism) is NP-complete. In addition, a new distance measure, called Extended Graph Edit Distance (EGED), is introduced in both non-metric and metric spaces for matching and indexing respectively. Based on STRG and EGED, we propose a new indexing method STRG-Index, which is faster and more accurate since it uses tree structure and clustering algorithm. We compare the STRG-Index with the M-tree, which is a popular tree-based indexing method for multimedia data. The STRG-Index outperforms the M-tree for various query loads in terms of cost and speed.

[1]  Alan Hanjalic,et al.  Automated high-level movie segmentation for advanced video-retrieval systems , 1999, IEEE Trans. Circuits Syst. Video Technol..

[2]  Lei Chen,et al.  Symbolic representation and retrieval of moving object trajectories , 2004, MIR '04.

[3]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[4]  John R. Smith,et al.  Modeling semantic concepts to support query by keywords in video , 2002, Proceedings. International Conference on Image Processing.

[5]  Ricardo A. Baeza-Yates,et al.  Searching in metric spaces , 2001, CSUR.

[6]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  G. Levi.  A note on the derivation of maximal common subgraphs of two directed or undirected graphs , 1973 .

[8]  Lei Chen,et al.  On The Marriage of Lp-norms and Edit Distance , 2004, VLDB.

[9]  Herbert Gish,et al.  Parametric trajectory models for speech recognition , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[10]  Svetha Venkatesh,et al.  Spatial Indexing for Video Databases , 1996, J. Vis. Commun. Image Represent..

[11]  Greg Hamerly,et al.  Alternatives to the k-means algorithm that find better clusterings , 2002, CIKM '02.

[12]  Dimitrios Gunopulos,et al.  Robust similarity measures for mobile object trajectories , 2002, Proceedings. 13th International Workshop on Database and Expert Systems Applications.

[13]  Timos K. Sellis,et al.  Spatio-temporal indexing for large multimedia applications , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[14]  Amir Averbuch,et al.  Tracking of moving objects based on graph edges similarity , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[15]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[16]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[17]  Roger Mohr,et al.  Probabilistic Hierarchical Framework for Clustering of Tracked Objects in Video Streams , 2000 .

[18]  Horst Bunke,et al.  Subgraph Isomorphism in Polynomial Time , 1995 .

[19]  Christos Faloutsos,et al.  Automatic multimedia cross-modal correlation discovery , 2004, KDD.

[20]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[21]  Kien A. Hua,et al.  Content-based scene change detection and classification technique using background tracking , 1999, Electronic Imaging.

[22]  Pavel Zezula,et al.  M-tree: An Efficient Access Method for Similarity Search in Metric Spaces , 1997, VLDB.

[23]  Vamsi K. Srikantam,et al.  On mixture density and maximum likelihood power estimation via expectation-maximization , 2000, ASP-DAC '00.

[24]  Anil K. Jain,et al.  Image retrieval using color and shape , 1996, Pattern Recognit..

[25]  László Böszörményi,et al.  VIDEX: an integrated generic video indexing approach , 2000, ACM Multimedia.

[26]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[27]  J. J. McGregor,et al.  Backtrack search algorithms and the maximal common subgraph problem , 1982, Softw. Pract. Exp..

[28]  Sergio A. Velastin,et al.  People tracking in surveillance applications , 2006, Image Vis. Comput..