Developing context model supporting spatial relations for semantic video retrieval

Video retrieval is one of the most famous issues in multimedia research. Users express their needs in terms of queries and expect to retrieve most relevant answers. This task is becoming harder due to large amount of video archives including broadcast news, documentary videos, movies and particularly internet which let users to search and share videos all over the world. Retrieving videos is challenging since queries can be on object, motion, texture, color, audio, etc. Indexing the video content supporting spatial relations will give more accurate results. For this purpose, segmentation and localization should be done. In this work, we propose to find new concept without availability of annotation and then retrieve videos according to queries defined by users and support spatial relations. To do mentioned purposes, we developed context model supporting spatial relations for better retrieval results. Our main contribution for better video retrieval is extracting spatial information in addition to using semantic word similarity for finding new concept without availability of annotations.

[1]  Max J. Egenhofer,et al.  Reasoning about Gradual Changes of Topological Relationships , 1992, Spatio-Temporal Reasoning.

[2]  M. Egenhofer,et al.  Point-Set Topological Spatial Relations , 2001 .

[3]  Markus A. Stricker,et al.  Color indexing with weak spatial constraints , 1996, Electronic Imaging.

[4]  Marcel Worring,et al.  The challenge problem for automated detection of 101 semantic concepts in multimedia , 2006, MM '06.

[5]  Shi-Kuo Chang,et al.  Representation And Retrieval Of Symbolic Pictures Using Generalized 2D Strings , 1989, Other Conferences.

[6]  Suh-Yin Lee,et al.  2D C-string: A new spatial knowledge representation for image database systems , 1990, Pattern Recognit..

[7]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[8]  Marcel Worring,et al.  Are Concept Detector Lexicons Effective for Video Search? , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[9]  Karen Spärck Jones,et al.  Automatic content-based retrieval of broadcast news , 1995, MULTIMEDIA '95.

[10]  B. S. Manjunath,et al.  A comparison of wavelet transform features for texture image annotation , 1995, Proceedings., International Conference on Image Processing.

[11]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[12]  Paul Over,et al.  TRECVID: Benchmarking the Effectivenss of Information Retrieval Tasks on Digital Video , 2003, CIVR.

[13]  Shi-Kuo Chang,et al.  An Intelligent Image Database System , 1988, IEEE Trans. Software Eng..

[14]  Shih-Fu Chang,et al.  Columbia University’s Baseline Detectors for 374 LSCOM Semantic Visual Concepts , 2007 .

[15]  Jiebo Luo,et al.  Utilizing semantic word similarity measures for video retrieval , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Chong-Wah Ngo,et al.  Towards optimal bag-of-features for object categorization and semantic video retrieval , 2007, CIVR '07.

[17]  Peter D. Turney.  Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL , 2001, ECML.

[18]  Lei Chen,et al.  A Multi-Level Index Structure for Video Databases , 2002, Multimedia Information Systems.

[19]  SUH-YIN LEE,et al.  Spatial reasoning and similarity retrieval of images using 2D C-string knowledge representation , 1992, Pattern Recognit..

[20]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[21]  Jin Zhao,et al.  Video Retrieval Using High Level Features: Exploiting Query Matching and Confidence-Based Weighting , 2006, CIVR.

[22]  Andrew U. Frank,et al.  Theories and Methods of Spatio-Temporal Reasoning in Geographic Space , 1992, Lecture Notes in Computer Science.

[23]  Carlo Strapparava,et al.  Corpus-based and Knowledge-based Measures of Text Semantic Similarity , 2006, AAAI.

[24]  Marcel Worring,et al.  Adding Semantics to Detectors for Video Retrieval , 2007, IEEE Transactions on Multimedia.

[25]  Timos K. Sellis,et al.  Spatio-temporal composition in multimedia applications , 1996, Proceedings International Workshop on Multimedia Software Development.

[26]  Haim H. Permuter,et al.  IBM Research TREC 2002 Video Retrieval System , 2002, TREC.

[27]  Rong Yan,et al.  Can High-Level Concepts Fill the Semantic Gap in Video Retrieval? A Case Study With Broadcast News , 2007, IEEE Transactions on Multimedia.

[28]  Maneesh Kumar Singh,et al.  State-of-the-art on spatio-temporal information-based video retrieval , 2009, Pattern Recognit..

[29]  Arbee L. P. Chen,et al.  3D-List: A Data Structure for Efficient Video Query Processing , 2002, IEEE Trans. Knowl. Data Eng..

[30]  Ricky K. Taira,et al.  A Knowledge-Based Approach for Retrieving Images by Content , 1996, IEEE Trans. Knowl. Data Eng..

[31]  Erland Jungert,et al.  Extended Symbolic Projections as a Knowledge Structure for Spatial Reasoning , 1988, Pattern Recognition.

[32]  TheodoridisYannis,et al.  Topological relations in the world of minimum bounding rectangles , 1995 .

[33]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Anoop Gupta,et al.  Automatically extracting highlights for TV Baseball programs , 2000, ACM Multimedia.

[35]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.