A Survey of Semantic Image and Video Annotation Tools

The availability of semantically annotated image and video assets constitutes a critical prerequisite for the realisation of intelligent knowledge management services pertaining to realistic user needs. Given the extend of the challenges involved in the automatic extraction of such descriptions, manually created metadata play a significant role, further strengthened by their deployment in training and evaluation tasks related to the automatic extraction of content descriptions. The different views taken by the two main approaches towards semantic content description, namely the Semantic Web and MPEG-7, as well as the traits particular to multimedia content due to the multiplicity of information levels involved, have resulted in a variety of image and video annotation tools, adopting varying description aspects. Aiming to provide a common framework of reference and furthermore to highlight open issues, especially with respect to the coverage and the interoperability of the produced metadata, in this chapter we present an overview of the state of the art in image and video annotation tools.

[1]  C. Saathoff,et al.  KAT: The K-Space Annotation Tool , 2008 .

[2]  Rainer Lienhart,et al.  The Holy Grail of Multimedia Information Retrieval: So Close or Yet So Far Away? , 2008 .

[3]  Steffen Staab,et al.  Semantic Multimedia , 2008, Reasoning Web.

[4]  Marcel Worring,et al.  Adding Semantics to Detectors for Video Retrieval , 2007, IEEE Transactions on Multimedia.

[5]  Jane Hunter,et al.  The ABC Ontology and Model , 2001, J. Digit. Inf..

[6]  Michael G. Strintzis,et al.  Capturing MPEG-7 Semantics , 2007, MTSR.

[7]  Steffen Staab,et al.  COMM: Designing a Well-Founded Multimedia Ontology for the Web , 2007, ISWC/ASWC.

[8]  Nicola Guarino,et al.  Sweetening Ontologies with DOLCE , 2002, EKAW.

[9]  Raphaël Troncy,et al.  Enabling Multimedia Metadata Interoperability by Defining Formal Semantics of MPEG-7 Profiles , 2006, SAMT.

[10]  Shih-Fu Chang,et al.  A conceptual framework and empirical research for classifying visual descriptors , 2001 .

[11]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[12]  Aldo Gangemi,et al.  Ontology Design Patterns for Semantic Web Content , 2005, SEMWEB.

[13]  Jane Hunter,et al.  Adding Multimedia to the Semantic Web: Building an MPEG-7 ontology , 2001, SWWS.

[14]  Michael Kipp,et al.  ANVIL - a generic annotation tool for multimodal dialogue , 2001, INTERSPEECH.

[15]  Steffen Staab,et al.  M-OntoMat-Annotizer: Image Annotation Linking Ontologies and Multimedia Low-Level Features , 2006, KES.

[16]  Chrisa Tsinaraki,et al.  Interoperability Support between MPEG-7/21 and OWL in DS-MIRF , 2007, IEEE Transactions on Knowledge and Data Engineering.

[17]  José M. Martínez.  Standards - MPEG-7 overview of MPEG-7 description tools, part 2 , 2002 .

[18]  Jane Hunter,et al.  Vannotea: A collaborative video indexing, annotation and discussion system for broadband networks , 2003 .

[19]  Lynda Hardman,et al.  That obscure object of desire: multimedia metadata on the Web, Part-1 , 2004, IEEE MultiMedia.

[20]  Michael Kipp.  Spatiotemporal Coding in ANVIL , 2008, LREC.

[21]  Simon Miles.  Electronically Querying for the Provenance of Entities , 2006, IPAW.

[22]  Roberto García,et al.  Semantic Integration and Retrieval of Multimedia Metadata , 2005, SemAnnot@ISWC.

[23]  Daniel L. Rubin,et al.  iPad: Semantic Annotation and Markup of Radiological Images , 2008, AMIA.

[24]  Mathias Lux,et al.  Caliph & Emir : Semantic Annotation and Retrieval in Personal Digital Photo Libraries , 2004 .

[25]  Marcel Worring,et al.  Classification of user image descriptions , 2004, Int. J. Hum. Comput. Stud..

[26]  Fabio Ciravegna,et al.  Cross-media document annotation and enrichment , 2006, SAAW@ISWC.

[27]  James A. Hendler,et al.  Annotation and Provenance Tracking in Semantic Web Photo Libraries , 2006, IPAW.

[28]  Ermanno Bencivenga.  That Obscure Object of Desire , 1988 .

[29]  Chrisa Tsinaraki,et al.  Integration of OWL Ontologies in MPEG-7 and TV-Anytime Compliant Semantic Indexing , 2004, CAiSE.

[30]  Lloyd Rutledge.  SMIL 2.0: XML for Web Multimedia , 2001, IEEE Internet Comput..

[31]  Enrico Motta,et al.  The Semantic Web - ISWC 2005, 4th International Semantic Web Conference, ISWC 2005, Galway, Ireland, November 6-10, 2005, Proceedings , 2005, SEMWEB.

[32]  Shih-Fu Chang,et al.  A conceptual framework and empirical research for classifying visual descriptors , 2001, J. Assoc. Inf. Sci. Technol..

[33]  Miguel Soriano,et al.  Soft-decision tracing in fingerprinted multimedia content , 2004, IEEE MultiMedia.

[34]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Rong Yan,et al.  How many high-level concepts will fill the semantic gap in news video retrieval? , 2007, CIVR '07.

[36]  Raphaël Troncy,et al.  MPEG-7 based Multimedia Ontologies: Interoperability Support or Interoperability Issue? , 2007 .

[37]  Michael Hausenblas,et al.  Why Real-World Multimedia Assets Fail to Enter the Semantic Web , 2007, SAAKM.

[38]  Peter Schallauer,et al.  Efficient Semantic Video Annoation by Object and Shot Re-Detection , 2008 .

[39]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[40]  Lakhmi C. Jain,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2004, Lecture Notes in Computer Science.

[41]  Raphaël Troncy,et al.  Deploying Multimedia Metadata on the Semantic Web , 2007, SAMT.

[42]  Ian Horrocks,et al.  Ontologies and the semantic web , 2008, CACM.

[43]  Raphaël Troncy,et al.  Describing low-level image features using the COMM ontology , 2008, 2008 15th IEEE International Conference on Image Processing.