An Ontology Framework For Knowledge-Assisted Semantic Video Analysis and Annotation

An approach for knowledge assisted semantic analysis and annotation of video content, based on an ontology infrastructure is presented. Semantic concepts in the context of the examined domain are defined in an ontology, enriched with qualitative attributes of the semantic objects (e.g. color homogeneity), multimedia processing methods (color clustering, respectively), and numerical data or low-level features generated via training (e.g. color models, also defined in the ontology). Semantic Web technologies are used for knowledge representation in RDF/RDFS language. Rules in F-logic are defined to describe how tools for multimedia analysis should be applied according to different object attributes and low-level features, aiming at the detection of video objects corresponding to the semantic concepts defined in the ontology. This supports flexible and managed execution of various application and domain independent multimedia analysis tasks. This ontology-based approach provides the means of generating semantic metadata and as a consequence Semantic Web services and applications have a greater chance of discovering and exploiting the information and knowledge in multimedia data. The proposed approach is demonstrated in the Formula One and Football domains and shows promising results.

[1]  Masahito Hirakawa,et al.  Knowledge-assisted content-based retrieval for multimedia databases , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[2]  Ferran Marqués,et al.  Region-based representations of image and video: segmentation tools for multimedia services , 1999, IEEE Trans. Circuits Syst. Video Technol..

[3]  Shih-Fu Chang,et al.  Overview of the MPEG-7 standard , 2001, IEEE Trans. Circuits Syst. Video Technol..

[4]  Steffen Staab,et al.  OntoEdit: Guiding Ontology Development by Methodology and Inferencing , 2002, OTM.

[5]  Michael G. Strintzis,et al.  Multimedia Content Indexing and Retrieval Using an Object Ontology , 2005, Multimedia Content and the Semantic Web.

[6]  Georg Lausen,et al.  Ontologies in F-logic , 2004, Handbook on Ontologies.

[7]  Michael Kifer,et al.  Logical foundations of object-oriented and frame-based languages , 1995, JACM.

[8]  Robert Tansley,et al.  Automating the linking of content and concept , 2000, ACM Multimedia.

[9]  Shih-Fu Chang,et al.  Image classification using multimedia knowledge networks , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[10]  Thomas S. Huang,et al.  Factor graph framework for semantic video indexing , 2002, IEEE Trans. Circuits Syst. Video Technol..

[11]  David Sinclair,et al.  A Self-Referential Perceptual Inference Framework for Video Interpretation , 2003, ICVS.

[12]  Shih-Fu Chang,et al.  The holy grail of content-based media analysis , 2002 .

[13]  Atsuo Yoshitaka,et al.  A Survey on Content-Based Retrieval for Multimedia Databases , 1999, IEEE Trans. Knowl. Data Eng..

[14]  Giorgos Stamou,et al.  Knowledge – Assisted Video Analysis and Object Detection , 2002 .

[15]  Michael G. Strintzis,et al.  An ontology approach to object-based image retrieval , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[16]  Yannis Avrithis,et al.  USING CONTEXT AND FUZZY RELATIONS TO INTERPRET MULTIMEDIA CONTENT , 2003 .

[17]  Arif Ghafoor,et al.  Semantic Modeling and Knowledge Representation in Multimedia Databases , 1999, IEEE Trans. Knowl. Data Eng..

[18]  Jane Hunter,et al.  Realizing the hydrogen economy through Semantic Web technologies , 2004, IEEE Intelligent Systems.

[19]  Michael G. Strintzis,et al.  Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval , 2004, IEEE Transactions on Circuits and Systems for Video Technology.