Semantic Representation of Multimedia Content

Multimedia documents constitute extremely rich information resources, whose efficacious management is intertwined with the effective capturing of the underlying semantics. The conveyed meaning may span along multiple levels and relates to search and retrieval tasks, much as to the very extraction and interpretation of content descriptions. In this chapter we consider the formal representation of multimedia semantics that pertain to media and domain specific descriptions, for the purpose of supporting both the extraction and subsequent semantic management of such descriptions. To this end, firstly, we present first an overview of existing approaches to the representation of multimedia content and discuss open issues. Subsequently, we present the ontology infrastructure developed in the context of the BOEMIE project tailored towards the formal representation of multimedia content. Concluding, we present what can non-standard formal representation technologies, such as fuzzy knowledge representation formalisms bring to multimedia document processing and management.

[1]  Heiner Stuckenschmidt,et al.  Handbook on Ontologies , 2004, Künstliche Intell..

[2]  Michael G. Strintzis,et al.  Investigating fuzzy DLs-based reasoning in semantic image analysis , 2010, Multimedia Tools and Applications.

[3]  Jane Hunter,et al.  Evaluating the application of semantic inferencing rules to image annotation , 2005, K-CAP '05.

[4]  Diego Calvanese,et al.  The Description Logic Handbook: Theory, Implementation, and Applications , 2003, Description Logic Handbook.

[5]  Anupriya Ankolekar,et al.  DOLCE ergo SUMO: On foundational and domain models in the SmartWeb Integrated Ontology (SWIntO) , 2007, J. Web Semant..

[6]  Bo Hu,et al.  Ontology-based medical image annotation with description logics , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[7]  Giorgos Stoilos,et al.  Extending Fuzzy Description Logics for the Semantic Web , 2007, OWLED.

[8]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[9]  Lynda Hardman,et al.  That obscure object of desire: multimedia metadata on the Web, Part-1 , 2004, IEEE MultiMedia.

[10]  José M. Martínez.  Standards - MPEG-7 overview of MPEG-7 description tools, part 2 , 2002 .

[11]  Ian Horrocks,et al.  Fuzzy OWL: Uncertainty and the Semantic Web , 2005, OWLED.

[12]  Michael G. Strintzis,et al.  Knowledge-assisted semantic video object detection , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Aldo Gangemi,et al.  Ontology Design Patterns for Semantic Web Content , 2005, SEMWEB.

[14]  Ian Horrocks,et al.  Practical Reasoning for Expressive Description Logics , 1999, LPAR.

[15]  Peter F. Patel-Schneider,et al.  Reducing OWL entailment to description logic satisfiability , 2004, Journal of Web Semantics.

[16]  Michael G. Strintzis,et al.  Multimedia Reasoning with Natural Language Support , 2007, International Conference on Semantic Computing (ICSC 2007).

[17]  Chrisa Tsinaraki,et al.  Integration of OWL Ontologies in MPEG-7 and TV-Anytime Compliant Semantic Indexing , 2004, CAiSE.

[18]  Susanne Boll,et al.  Multimedia Vocabularies on the Semantic Web , 2007 .

[19]  Lynda Hardman,et al.  That Obscure Object of Desire: Multimedia Metadata on the Web, Part 1 , 2004, IEEE Multim..

[20]  Thomas Lukasiewicz,et al.  P-SHOQ(D): A Probabilistic Extension of SHOQ(D) for Probabilistic Ontologies in the Semantic Web , 2002, JELIA.

[21]  Bernd Neumann,et al.  Ontology-Based Reasoning Techniques for Multimedia Interpretation and Retrieval , 2008 .

[22]  Bob J. Wielinga,et al.  Ontology-Based Photo Annotation , 2001, IEEE Intell. Syst..

[23]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[24]  Thomas R. Gruber,et al.  Toward principles for the design of ontologies used for knowledge sharing? , 1995, Int. J. Hum. Comput. Stud..

[25]  Steffen Staab,et al.  COMM: Designing a Well-Founded Multimedia Ontology for the Web , 2007, ISWC/ASWC.

[26]  George J. Klir,et al.  Fuzzy sets and fuzzy logic - theory and applications , 1995 .

[27]  Michael G. Strintzis,et al.  Enquiring MPEG-7 based multimedia ontologies , 2009, Multimedia Tools and Applications.

[28]  Werner Nutt,et al.  Basic Description Logics , 2003, Description Logic Handbook.

[29]  Francesco M. Donini,et al.  Description Logics for Image Recognition: a preliminary proposal , 1999, Description Logics.

[30]  José María Martínez Sanchez.  MPEG-7: Overview of MPEG-7 Description Tools, Part 2 , 2002, IEEE Multim..

[31]  Lynda Hardman,et al.  That Obscure Object of Desire: Multimedia on the Web, Part 2 , 2005 .

[32]  Raphaël Troncy,et al.  MPEG-7 based Multimedia Ontologies: Interoperability Support or Interoperability Issue? , 2007 .

[33]  Miguel Soriano,et al.  Soft-decision tracing in fingerprinted multimedia content , 2004, IEEE MultiMedia.

[34]  Eero Hyvönen,et al.  Ontology-Based Image Retrieval , 2003, WWW.

[35]  Bernd Neumann,et al.  On scene interpretation with description logics , 2006, Image Vis. Comput..

[36]  Alberto Del Bimbo,et al.  Enhanced ontologies for video annotation and retrieval , 2005, MIR '05.

[37]  Yves Chiaramella,et al.  A Model for Multimedia Information Retrieval , 1996 .

[38]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Ermanno Bencivenga.  That Obscure Object of Desire , 1988 .

[40]  Umberto Straccia,et al.  Reasoning within Fuzzy Description Logics , 2011, J. Artif. Intell. Res..

[41]  Chrisa Tsinaraki,et al.  Interoperability Support between MPEG-7/21 and OWL in DS-MIRF , 2007, IEEE Transactions on Knowledge and Data Engineering.

[42]  G. Stamou,et al.  Reasoning with Very Expressive Fuzzy Description Logics , 2007, J. Artif. Intell. Res..

[43]  Umberto Straccia,et al.  Towards a Fuzzy Description Logic for the Semantic Web (Preliminary Report) , 2005, ESWC.

[44]  Marcel Worring,et al.  Building a visual ontology for video retrieval , 2005, MULTIMEDIA '05.

[45]  Jane Hunter,et al.  Realizing the hydrogen economy through Semantic Web technologies , 2004, IEEE Intelligent Systems.

[46]  John R. Smith,et al.  Modal Keywords, Ontologies, and Reasoning for Video Understanding , 2003, CIVR.

[47]  Jianfeng Du,et al.  Extending description logics with uncertainty reasoning in possibilistic logic , 2007, Int. J. Intell. Syst..

[48]  Roberto García,et al.  Semantic Integration and Retrieval of Multimedia Metadata , 2005, SemAnnot@ISWC.

[49]  Monique Thonnat,et al.  A cognitive vision platform for automatic recognition of natural complex objects , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[50]  Monique Thonnat,et al.  A Weakly Supervised Approach for Semantic Image Indexing and Retrieval , 2005, CIVR.

[51]  Steffen Staab,et al.  Semantic Annotation of Images and Videos for Multimedia Analysis , 2005, ESWC.

[52]  Jane Hunter,et al.  Adding Multimedia to the Semantic Web: Building an MPEG-7 ontology , 2001, SWWS.

[53]  Steffen Staab,et al.  M-OntoMat-Annotizer: Image Annotation Linking Ontologies and Multimedia Low-Level Features , 2006, KES.

[54]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[55]  Paul H. Lewis,et al.  SCULPTEUR: Towards a New Paradigm for Multimedia Museum Information Handling , 2003, SEMWEB.

[56]  Yun Peng,et al.  A probabilistic extension to ontology language OWL , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[57]  Huajun Chen,et al.  The Semantic Web , 2011, Lecture Notes in Computer Science.

[58]  Nicola Guarino,et al.  Sweetening Ontologies with DOLCE , 2002, EKAW.

[59]  Otthein Herzog,et al.  Content-based Image Retrieval by Ontology-based Object Recognition , 2004 .

[60]  Stefanos D. Kollias,et al.  Multimedia Reasoning with f-SHIN , 2007, Second International Workshop on Semantic Media Adaptation and Personalization (SMAP 2007).

[61]  Aldo Gangemi,et al.  Understanding the Semantic Web through Descriptions and Situations , 2003, OTM.

[62]  Deborah L. McGuinness,et al.  OWL Web ontology language overview , 2004 .