Building a visual ontology for video retrieval

To ensure access to growing video collections, annotation is becoming more and more important using background knowledge in the form of ontologies or thesauri is a way to facilitate annotation in a broad domain. Current ontologies are not suitable for (semi-) automatic annotation of visual resources as they contain little visual information about the concepts they describe. We investigate how an ontology that does contain visual information can facilitate annotation in a broad domain and identify requirements that a visual ontology has to meet. Based on these requirements, we create a visual ontology out of two existing knowledge corpora (WordNet and MPEG-7) by creating links between visual and general concepts. We test performance of the ontology on 40 shots of news video, and discuss the added value of each visual property.

[1]  C. Fellbaum.  An Electronic Lexical Database , 1998 .

[2]  Jane Hunter,et al.  Adding Multimedia to the Semantic Web: Building an MPEG-7 ontology , 2001, SWWS.

[3]  Arnold W. M. Smeulders,et al.  c ○ 2005 Springer Science + Business Media, Inc. Manufactured in The Netherlands. A Six-Stimulus Theory for Stochastic Texture , 2002 .

[4]  Frank van Harmelen,et al.  A semantic web primer , 2004 .

[5]  Anthony Hoogs,et al.  Video content annotation using visual analysis and a large semantic knowledgebase , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  Steffen Staab,et al.  Annotation for the semantic web , 2003 .

[7]  Alexander G. Hauptmann,et al.  Towards a Large Scale Concept Ontology for Broadcast Video , 2004, CIVR.

[8]  Bob J. Wielinga,et al.  Ontology-Based Photo Annotation , 2001, IEEE Intell. Syst..

[9]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Dennis Koelma,et al.  The MediaMill TRECVID 2008 Semantic Video Search Engine , 2008, TRECVID.

[11]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[12]  José María Martínez Sanchez,et al.  Towards universal access to content using MPEG-7 , 2002, MULTIMEDIA '02.

[13]  A. T. Schreiber,et al.  Semantic Annotation of Image Collections , 2003 .

[14]  Anthony Hoogs,et al.  Enabling video annotation using a semantic database extended with visual knowledge , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[15]  Arnold W. M. Smeulders,et al.  Color texture measurement and segmentation , 2005, Signal Process..

[16]  Guus Schreiber,et al.  The Semantic Web – ISWC 2004 , 2004, Lecture Notes in Computer Science.

[17]  Shih-Fu Chang,et al.  Overview of the MPEG-7 standard , 2001, IEEE Trans. Circuits Syst. Video Technol..

[18]  Michael G. Strintzis,et al.  Region-Based Image Retrieval Using an Object Ontology and Relevance Feedback , 2004, EURASIP J. Adv. Signal Process..

[19]  Eero Hyvönen,et al.  A Cultural Community Portal for Publishing Museum Collections on the Semantic Web , 2004, ECAI Workshop on Application of Semantic Web Technologies to Web Communities.

[20]  Edmund Lee.  Building Interoperability for United Kingdom Historic Environment Information Resources , 2004, ECDL.