Bounded coordinate system indexing for real-time video clip search

Recently, video clips have become very popular online. The massive influx of video clips has created an urgent need for video search engines to facilitate retrieving relevant clips. Different from traditional long videos, a video clip is a short video often expressing a moment of significance. Due to the high complexity of video data, efficient video clip search from large databases turns out to be very challenging. We propose a novel video clip representation model called the Bounded Coordinate System (BCS), which is the first single representative capturing the dominating content and content—changing trends of a video clip. It summarizes a video clip by a coordinate system, where each of its coordinate axes is identified by principal component analysis (PCA) and bounded by the range of data projections along the axis. The similarity measure of BCS considers the operations of translation, rotation, and scaling for coordinate system matching. Particularly, rotation and scaling reflect the difference of content tendencies. Compared with the quadratic time complexity of existing methods, the time complexity of measuring BCS similarity is linear. The compact video representation together with its linear similarity measure makes real-time search from video clip collections feasible. To further improve the retrieval efficiency for large video databases, a two-dimensional transformation method called Bidistance Transformation (BDT) is introduced to utilize a pair of optimal reference points with respect to bidirectional axes in BCS. Our extensive performance study on a large database of more than 30,000 video clips demonstrates that BCS achieves very high search accuracy according to human judgment. This indicates that content tendencies are important in determining the meanings of video clips and confirms that BCS can capture the inherent moment of video clip to some extent that better resembles human perception. In addition, BDT outperforms existing indexing methods greatly. Integration of the BCS model and BDT indexing can achieve real-time search from large video clip databases.

[1]  Donald A. Adjeroh,et al.  A Distance Measure for Video Sequences , 1999, Comput. Vis. Image Underst..

[2]  Chia-Wen Lin,et al.  Fast coarse-to-fine video retrieval using shot-level spatio-temporal statistics , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[4]  Avideh Zakhor,et al.  Fast similarity search and clustering of video sequences on the world-wide-web , 2005, IEEE Transactions on Multimedia.

[5]  Ramesh R. Sarukkai,et al.  Video search: opportunities & challenges , 2005, MIR '05.

[6]  Beng Chin Ooi,et al.  iDistance: An adaptive B+-tree based indexing method for nearest neighbor search , 2005, TODS.

[7]  Zi Huang,et al.  Statistical summarization of content features for fast near-duplicate video detection , 2007, ACM Multimedia.

[8]  Lei Chen,et al.  Robust and fast similarity search for moving object trajectories , 2005, SIGMOD '05.

[9]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Kohji Fukunaga,et al.  Introduction to Statistical Pattern Recognition-Second Edition , 1990 .

[11]  Christian Böhm,et al.  Searching in high-dimensional spaces: Index structures for improving the performance of multimedia databases , 2001, CSUR.

[12]  Laurent Amsaleg,et al.  Videntifier: identifying pirated videos in real-time , 2007, ACM Multimedia.

[13]  Jun Sakuma,et al.  Fast approximate similarity search in extremely high-dimensional data sets , 2005, 21st International Conference on Data Engineering (ICDE'05).

[14]  Beng Chin Ooi,et al.  Towards effective indexing for very large video sequence database , 2005, SIGMOD '05.

[15]  Changick Kim,et al.  Spatiotemporal sequence matching for efficient video copy detection , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Rakesh Mohan,et al.  Video sequence matching , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[17]  Yuxin Peng,et al.  Clip-based similarity measure for query-dependent clip retrieval and video summarization , 2006, IEEE Trans. Circuits Syst. Video Technol..

[18]  B. Vasudev,et al.  Spatiotemporal sequence matching for efficient video copy detection , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Rainer Lienhart,et al.  Comparison of automatic shot boundary detection algorithms , 1998, Electronic Imaging.

[20]  Sang Uk Lee,et al.  Efficient video indexing scheme for content-based retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[21]  Chu-Song Chen,et al.  A Time Warping Based Approach for Video Copy Detection , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[22]  Simone Santini,et al.  Similarity Measures , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[24]  Li Chen,et al.  Video copy detection: a comparative study , 2007, CIVR '07.

[25]  Yueting Zhuang,et al.  A new approach to retrieve video by example video clip , 1999, MULTIMEDIA '99.

[26]  Dimitrios Gunopulos,et al.  Discovering similar multidimensional trajectories , 2002, Proceedings 18th International Conference on Data Engineering.

[27]  Joachim M. Buhmann,et al.  Empirical evaluation of dissimilarity measures for color and texture , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[28]  Chong-Wah Ngo,et al.  Practical elimination of near-duplicates from web video search , 2007, ACM Multimedia.

[29]  Sharad Mehrotra,et al.  Local Dimensionality Reduction: A New Approach to Indexing High Dimensional Spaces , 2000, VLDB.

[30]  Jianping Fan,et al.  Exploring video content structure for hierarchical summarization , 2004, Multimedia Systems.

[31]  Tat-Seng Chua,et al.  A match and tiling approach to content-based video retrieval , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[32]  Alberto Del Bimbo,et al.  Video Clip Matching Using MPEG-7 Descriptors and Edit Distance , 2006, CIVR.

[33]  A. Murat Tekalp,et al.  Two-stage hierarchical video summary extraction to match low-level user browsing preferences , 2003, IEEE Trans. Multim..

[34]  Avideh Zakhor,et al.  Efficient video similarity measurement with video signature , 2002, Proceedings. International Conference on Image Processing.

[35]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[36]  Giridharan Iyengar,et al.  Distributional clustering for efficient content-based retrieval of images and video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[37]  Kunio Kashino,et al.  A quick search method for audio and video signals based on histogram pruning , 2003, IEEE Trans. Multim..

[38]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[39]  Ruud M. Bolle,et al.  Comparison of sequence matching techniques for video copy detection , 2001, IS&T/SPIE Electronic Imaging.

[40]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[41]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[42]  Jung-Hwan Oh,et al.  STRG-Index: spatio-temporal region graph indexing for large video databases , 2005, SIGMOD '05.

[43]  Alessandra Lumini,et al.  MKL-tree: an index structure for high-dimensional vector spaces , 2007, Multimedia Systems.

[44]  Yueting Zhuang,et al.  Content-based video similarity model , 2000, ACM Multimedia.

[45]  Xiao-Ping Zhang,et al.  Automatic identification of digital video based on shot-level sequence matching , 2005, MULTIMEDIA '05.

[46]  Justin Zobel,et al.  Detection of video sequences using compact signatures , 2006, TOIS.

[47]  Hans-Peter Kriegel,et al.  The pyramid-technique: towards breaking the curse of dimensionality , 1998, SIGMOD '98.

[48]  HuangZi,et al.  Bounded coordinate system indexing for real-time video clip search , 2009 .

[49]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[50]  Nicu Sebe,et al.  Toward Improved Ranking Metrics , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Lei Chen,et al.  MINDEX: An efficient index structure for salient-object-based queries in video databases , 2004, Multimedia Systems.

[52]  Kenneth Rose,et al.  VQ-index: an index structure for similarity searching in multimedia databases , 2002, MULTIMEDIA '02.

[53]  John P. Oakley,et al.  Storage and Retrieval for Image and Video Databases , 1993 .

[54]  Milind R. Naphade,et al.  Novel scheme for fast and efficent video sequence matching using compact signatures , 1999, Electronic Imaging.

[55]  Deok-Hwan Kim,et al.  Similarity search for multidimensional data sequences , 2000, Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073).

[56]  David C. Gibbon,et al.  Introduction to video search engines , 2008 .

[57]  Hans-Peter Kriegel,et al.  The X-tree : An Index Structure for High-Dimensional Data , 2001, VLDB.

[58]  Heng Tao Shen,et al.  Exploring composite acoustic features for efficient music similarity query , 2006, MM '06.

[59]  Hans-Jörg Schek,et al.  A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces , 1998, VLDB.

[60]  Joachim M. Buhmann,et al.  Empirical Evaluation of Dissimilarity Measures for Color and Texture , 2001, Comput. Vis. Image Underst..

[61]  I. Jolliffe.  Principal Component Analysis , 2002 .