A Collaborative Video Annotation System Based on Semantic Web Technologies

In recent years, videos have become more and more a familiar multimedia format for common users. In particular, the advent of Web 2.0 and the spreading of video-sharing services over the Web have led to an explosion of online video content. The capability to provide broader support in accessing and exploring video content, and in general other kind of multimedia formats as images and documents, is becoming more and more important. In this context, the value of semantically structured data and metadata is recognized as a key factor both to improve search efficiency and to guarantee data interoperability. This latter aspect is critical to connect different, heterogeneous content coming from a variety of data sources. On the other hand, the annotation of video resources has been increasingly understood as a medium factor to enable deep analysis of contents and collaborative study of online digital objects. However, as existing annotation tools provide poor support for semantically structured content or in some cases express the semantics in proprietary and non-interoperable formats, such knowledge that users build by carefully annotating contents hardly crosses the boundaries of a single system and often cannot be reused by different communities (e.g., to classify content or to discover new relations among resources). In this paper, a novel Semantic Web-based annotation system is presented that enables user annotations to form semantically structured knowledge at different levels of granularity and complexity. Annotation can be reused by external applications and mixed with Web of Data sources to enable “serendipity,” the reuse of data produced for a specific task (annotation) by different people and in different contexts from the one data originated from. The main ideas behind the approach are to build on ontologies and support linking, at data level, to precise thesauri and vocabularies, as well as to the Linked Open Data cloud. By describing the software model, developed in the context of SemLib EU project, and by providing an implementation of an online video annotation tool, the main aim of this paper is to demonstrate how such technologies can enable a scenario where users annotations are created while browsing the Web, naturally shared among users, stored in machine readable format and then possibly recombined with external data and ontologies to enhance end-user experience.

[1]  Rose Holley,et al.  Crowdsourcing: How and Why Should Libraries Do It? , 2010, D Lib Mag..

[2]  Francesco Piazza,et al.  Towards Semantic Multimodal Video Annotation , 2010, COST 2102 Training School.

[3]  Wenfei Fan,et al.  Keys for XML , 2001, WWW '01.

[4]  Li Li,et al.  A Survey on Visual Content-Based Video Indexing and Retrieval , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Francesco Piazza,et al.  Sentic Web: A New Paradigm for Managing Social Media Affective Information , 2011, Cognitive Computation.

[6]  Thomas C. Schmidt Transcribing and annotating spoken language with EXMARaLDA , 2004 .

[7]  Erik Cambria,et al.  Sentic Computing: Techniques, Tools, and Applications , 2012 .

[8]  Jens Lehmann,et al.  DBpedia - A crystallization point for the Web of Data , 2009, J. Web Semant..

[9]  Georgios C. Anagnostopoulos,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2003, Lecture Notes in Computer Science.

[10]  Marco Grassi Developing HEO Human Emotions Ontology , 2009, COST 2101/2102 Conference.

[11]  Andrea Ferracani,et al.  Sirio, orione and pan: an integrated web system for ontology-based video search and annotation , 2010, ACM Multimedia.

[12]  Christian Morbidoni,et al.  Introducing the Semlib Project: Semantic Web Tools for Digital Libraries , 2011, SDA.

[13]  Peter Wittenburg,et al.  OntoELAN: an ontology-based linguistic multimedia annotator , 2004, IEEE Sixth International Symposium on Multimedia Software Engineering.

[14]  Marja-Riitta Koivunen,et al.  Annotea: an open RDF infrastructure for shared Web annotations , 2001, WWW '01.

[15]  Bernhard Haslhofer,et al.  Augmenting Europeana content with linked data resources , 2010, I-SEMANTICS '10.

[16]  Christian Morbidoni,et al.  Semantic Web Techniques Application for Video Fragment Annotation and Management , 2010, COST 2102 Conference.

[17]  Michael Kipp,et al.  ANVIL - a generic annotation tool for multimodal dialogue , 2001, INTERSPEECH.

[18]  Pierre Andrews,et al.  A classification of semantic annotation systems , 2012, Semantic Web.

[19]  Jan-Torsten Milde,et al.  Comparison of multimodal annotation tools , 2006 .

[20]  Peter Wittenburg,et al.  ELAN: a Professional Framework for Multimodality Research , 2006, LREC.

[21]  Steffen Staab,et al.  M-OntoMat-Annotizer: Image Annotation Linking Ontologies and Multimedia Low-Level Features , 2006, KES.

[22]  Yiannis Kompatsiaris,et al.  A Survey of Semantic Image and Video Annotation Tools , 2011, Knowledge-Driven Multimedia Information Extraction and Ontology Evolution.

[23]  Jan-Torsten Milde,et al.  Comparison of multimodal annotation tools: Workshop report , 2006 .