Efficient video similarity measurement and search

We consider the use of meta-data and/or video-domain methods to detect similar videos on the Web. Meta-data is extracted from the textual and hyperlink information associated with each video clip. In the video domain, we apply an efficient similarity detection algorithm called video signature. The idea is to form a signature for each clip by selecting a small number of its frames that are most similar to a set of random seed images. We then apply a statistical pruning algorithm to allow fast detection on very large databases. Using a small ground-truth set, we achieve 90% recall and 95% precision using only 8% of the total number of operations required without pruning. For a database of around 46,000 video clips crawled from the Web, the video signature technique significantly outperforms meta-data in precision and recall. We show that even better performance can be achieved by combining them together. Based on our measurements, each video clip in our database has, on average, 1.53 similar copies.

[1]  A. Murat Tekalp,et al.  Hierarchical temporal video segmentation and content characterization , 1997, Other Conferences.

[2]  Avideh Zakhor,et al.  Estimation of Web video multiplicity , 1999, Electronic Imaging.

[3]  Giles,et al.  Searching the world wide Web , 1998, Science.

[4]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Ruud M. Bolle,et al.  Feature based indexing for media tracking , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[6]  Seth Pettie,et al.  An Optimal Minimum Spanning Tree Algorithm , 2000, ICALP.

[7]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[8]  Simone Santini,et al.  Similarity Measures , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Avideh Zakhor,et al.  Video similarity detection with video signature clustering , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[10]  Milind R. Naphade,et al.  Novel scheme for fast and efficent video sequence matching using compact signatures , 1999, Electronic Imaging.

[11]  Stanley Lemeshow,et al.  Sampling of Populations: Methods and Applications , 1991 .

[12]  H. Gabriela,et al.  Cluster-preserving Embedding of Proteins , 1999 .

[13]  Alan Hanjalic,et al.  Automated high-level movie segmentation for advanced video-retrieval systems , 1999, IEEE Trans. Circuits Syst. Video Technol..

[14]  Geoffrey Zweig,et al.  Syntactic Clustering of the Web , 1997, Comput. Networks.

[15]  C.-C. Jay Kuo,et al.  Hierarchical clustering techniques for image database organization and summarization , 1998, Other Conferences.

[16]  Boon-Lock Yeo,et al.  Segmentation of Video by Clustering and Graph Analysis , 1998, Comput. Vis. Image Underst..

[17]  Azriel Rosenfeld,et al.  Content-Based Access to Multimedia Information: From Technology Trends to State of the Art , 1999 .

[18]  Robin Sibson,et al.  SLINK: An Optimally Efficient Algorithm for the Single-Link Cluster Method , 1973, Comput. J..

[19]  Hector Garcia-Molina,et al.  Detecting Digital Copyright Violations On The Internet , 1999 .

[20]  Ellen M. Voorhees,et al.  Overview of the Seventh Text REtrieval Conference , 1998 .

[21]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.

[22]  Nuno Vasconcelos,et al.  On the complexity of probabilistic image retrieval , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[23]  Henry A. Kautz,et al.  Hardening soft information sources , 2000, KDD '00.

[24]  R. Motwani,et al.  High-Dimensional Computational Geometry , 2000 .

[25]  Linda G. Shapiro,et al.  A Flexible Image Database System for Content-Based Retrieval , 1999, Comput. Vis. Image Underst..

[26]  Fionn Murtagh,et al.  Comments on 'Parallel Algorithms for Hierarchical Clustering and Cluster Validity' , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Hanan Samet,et al.  The Design and Analysis of Spatial Data Structures , 1989 .

[28]  Hans-Jörg Schek,et al.  A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces , 1998, VLDB.

[29]  Jon M. Kleinberg,et al.  Two algorithms for nearest-neighbor search in high dimensions , 1997, STOC '97.

[30]  Rakesh Mohan,et al.  Video sequence matching , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[31]  Franz Aurenhammer,et al.  Voronoi diagrams—a survey of a fundamental geometric data structure , 1991, CSUR.

[32]  Michael J. Swain Searching for multimedia on the World Wide Web , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[33]  Ravi Kumar,et al.  Trawling the Web for Emerging Cyber-Communities , 1999, Comput. Networks.

[34]  Robert M. Haralick,et al.  Graph-theoretic clustering for image grouping and retrieval , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[35]  Roded Sharan,et al.  Algorithmic approaches to clustering gene expression data , 2001 .

[36]  Giridharan Iyengar,et al.  Distributional clustering for efficient content-based retrieval of images and video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[37]  Mohamed Abdel-Mottaleb,et al.  Image browsing using hierarchical clustering , 1999, Proceedings IEEE International Symposium on Computers and Communications (Cat. No.PR00250).

[38]  Milind R. Naphade,et al.  Multimodal pattern matching for audio-visual query and retrieval , 2001, IS&T/SPIE Electronic Imaging.

[39]  Gene H. Golub,et al.  Matrix computations , 1983 .

[40]  Donald A. Adjeroh,et al.  A distance measure for video sequence similarity matching , 1998, Proceedings International Workshop on Multi-Media Database Management Systems (Cat. No.98TB100249).

[41]  J. Bourgain On lipschitz embedding of finite metric spaces in Hilbert space , 1985 .

[42]  Wolfgang Effelsberg,et al.  On the detection and recognition of television commercials , 1997, Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[43]  John M. Gauch,et al.  Real Time Video Scene Detection and Classification , 1999, Inf. Process. Manag..

[44]  Richard L. Scheaffer,et al.  Elementary Survey Sampling , 1971 .

[45]  Shih-Fu Chang,et al.  VideoQ: a fully automated video retrieval system using motion sketches , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[46]  Alex Woronow Generating random numbers on a simplex , 1993 .

[47]  Iwan Setyawan,et al.  Watermarking digital image and video data. A state-of-the-art overview , 2000 .

[48]  Charles Elkan,et al.  An Efficient Domain-Independent Algorithm for Detecting Approximately Duplicate Database Records , 1997, DMKD.

[49]  Paul M. Aoki Generalizing "search" in generalized search trees , 1998, Proceedings 14th International Conference on Data Engineering.

[50]  Ian H. Witten,et al.  Managing Gigabytes: Compressing and Indexing Documents and Images , 1999 .

[51]  Ruud M. Bolle,et al.  Comparison of sequence matching techniques for video copy detection , 2001, IS&T/SPIE Electronic Imaging.

[52]  A E Bostwick,et al.  THE THEORY OF PROBABILITIES. , 1896, Science.

[53]  Mohan S. Kankanhalli,et al.  Content-based representative frame extraction for digital video , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[54]  G. Grimmett,et al.  Probability and random processes , 2002 .

[55]  Craig Silverstein,et al.  Analysis of a Very Large Altavista Query Log" SRC Technical note #1998-14 , 1998 .

[56]  Martin Farach-Colton,et al.  COFE: A Scalable Method for Feature Extraction from Complex Objects , 2000, DaWaK.

[57]  Pavel Zezula,et al.  M-tree: An Efficient Access Method for Similarity Search in Metric Spaces , 1997, VLDB.

[58]  Andrei Z. Broder,et al.  Mirror, Mirror on the Web: A Study of Host Pairs with Replicated Content , 1999, Comput. Networks.

[59]  C. J. van Rijsbergen,et al.  Report on the need for and provision of an 'ideal' information retrieval test collection , 1975 .

[60]  Wolfgang Effelsberg,et al.  VisualGREP: a systematic method to compare and retrieve video sequences , 1997, Electronic Imaging.

[61]  Rafail Ostrovsky,et al.  Efficient search for approximate nearest neighbor in high dimensional spaces , 1998, STOC '98.

[62]  Sanghyun Park,et al.  Indexing technique for similarity matching in large video databases , 2001, IS&T/SPIE Electronic Imaging.

[63]  Christos Faloutsos,et al.  FastMap: a fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets , 1995, SIGMOD '95.

[64]  Andrei Z. Broder,et al.  Graph structure in the Web , 2000, Comput. Networks.

[65]  Avideh Zakhor,et al.  Content analysis of video using principal components , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[66]  Béla Bollobás,et al.  Random Graphs , 1985 .

[67]  J. Kruskal On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[68]  Hector Garcia-Molina,et al.  Finding near-replicas of documents on the Web , 1999 .

[69]  Takeo Kanade,et al.  Video skimming and characterization through the combination of image and language understanding , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[70]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[71]  Andrei Z. Broder,et al.  A Technique for Measuring the Relative Size and Overlap of Public Web Search Engines , 1998, Comput. Networks.

[72]  Arnold W. M. Smeulders,et al.  Image Retrieval by Multi-scale Illumination Invariant Indexing , 1998, Multimedia Information Analysis and Retrieval.

[73]  James C. French,et al.  Using the triangle inequality to reduce the number of comparisons required for similarity-based retrieval , 1996, Electronic Imaging.

[74]  Brian Everitt,et al.  Cluster analysis , 1974 .

[75]  Christos Faloutsos,et al.  Searching Multimedia Databases by Content , 1996, Advances in Database Systems.

[76]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[77]  H. Hotelling Analysis of a complex of statistical variables into principal components. , 1933 .

[78]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[79]  Andrei Z. Broder,et al.  A Comparison of Techniques to Find Mirrored Hosts on the WWW , 2000, IEEE Data Eng. Bull..

[80]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .