Assessing concept selection for video retrieval

We explore the use of benchmarks to address the problem of assessing concept selection in video retrieval systems. Two benchmarks are presented, one created by human association of queries to concepts, the other generated from an extensively tagged collection. They are compared in terms of reliability, captured semantics, and retrieval performance. Recommendations are given for using the benchmarks to assess concept selection algorithms; the assessment is demonstrated on two existing algorithms. The benchmarks are released to the research community.

[1]  John R. Smith,et al.  A web-based system for collaborative annotation of large image and video collections: an evaluation and user study , 2005, MULTIMEDIA '05.

[2]  Chong-Wah Ngo,et al.  Ontology-enriched semantic space for video search , 2007, ACM Multimedia.

[3]  Rong Yan,et al.  Semantic concept-based query expansion and re-ranking for multimedia retrieval , 2007, ACM Multimedia.

[4]  Alan F. Smeaton.  Techniques used and open challenges to the analysis, indexing and retrieval of digital video , 2007, Inf. Syst..

[5]  Rong Yan,et al.  Can High-Level Concepts Fill the Semantic Gap in Video Retrieval? A Case Study With Broadcast News , 2007, IEEE Transactions on Multimedia.

[6]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[7]  Paul Over,et al.  TRECVID 2005 - An Overview , 2005, TRECVID.

[8]  Thomas Luckmann,et al.  The Structures of the Life World V1 Op , 1973 .

[9]  Mario F. Triola,et al.  Essentials of Statistics , 2001 .

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  John R. Smith,et al.  Large-scale concept ontology for multimedia , 2006, IEEE MultiMedia.

[12]  John R. Smith,et al.  Cluster-based data modeling for semantic video search , 2007, CIVR '07.

[13]  Wei-Hao Lin,et al.  Which Thousand Words are Worth a Picture? Experiments on Video Retrieval using a Thousand Concepts , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[14]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[15]  J. Fleiss.  Statistical methods for rates and proportions , 1974 .

[16]  Stephen E. Robertson,et al.  Okapi at TREC-3 , 1994, TREC.

[17]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[18]  Rong Yan,et al.  A review of text and image retrieval approaches for broadcast news video , 2007, Information Retrieval.

[19]  J. Fleiss,et al.  The measurement of interrater agreement , 2004 .

[20]  Marcel Worring,et al.  The challenge problem for automated detection of 101 semantic concepts in multimedia , 2006, MM '06.

[21]  Alexander G. Hauptmann,et al.  The Use and Utility of High-Level Semantic Features in Video Retrieval , 2005, CIVR.

[22]  George Lakoff,et al.  Women, Fire, and Dangerous Things , 1987 .

[23]  Jin Zhao,et al.  Video Retrieval Using High Level Features: Exploiting Query Matching and Confidence-Based Weighting , 2006, CIVR.

[24]  Dong Wang,et al.  The importance of query-concept-mapping for automatic video retrieval , 2007, ACM Multimedia.

[25]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[26]  Marcel Worring,et al.  Adding Semantics to Detectors for Video Retrieval , 2007, IEEE Transactions on Multimedia.

[27]  Marcel Worring,et al.  Semantic Image and Video Indexing in Broad Domains , 2007, IEEE Trans. Multim..