Towards Semantic Multimodal Video Annotation

Nowadays Semantic Web techniques are finding applications in several research fields. We believe that they can be beneficial also in multimodal video annotation to enhance the annotations management and to promote an effective sharing of collected multimodal data and annotations. To have an insight about how the task of video annotation is commonly performed and the created annotations are managed and to evaluate how to improve these tasks using semantic web techniques, we set up a publically available survey. In this paper, we discuss the results of the survey and trace a roadmap towards the application of semantic web techniques for the management of multimodal video annotations.

[1]  Lakhmi C. Jain,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2004, Lecture Notes in Computer Science.

[2]  Anton Nijholt,et al.  Development of Multimodal Interfaces: Active Listening and Synchrony, Second COST 2102 International Training School, Dublin, Ireland, March 23-27, 2009, Revised Selected Papers , 2010, COST 2102 Training School.

[3]  Peter Wittenburg,et al.  ELAN: a Professional Framework for Multimodality Research , 2006, LREC.

[4]  Erik Cambria,et al.  Sentic Computing: Exploitation of Common Sense for the Development of Emotion-Sensitive Systems , 2009, COST 2102 Training School.

[5]  Frank van Harmelen,et al.  A semantic web primer , 2004 .

[6]  Thomas C. Schmidt.  Transcribing and annotating spoken language with EXMARaLDA , 2004 .

[7]  Nicu Sebe,et al.  Multimodal approaches for emotion recognition: a survey , 2005, IS&T/SPIE Electronic Imaging.

[8]  Costanza Navarretta,et al.  The MUMIN multimodal coding scheme , 2005 .

[9]  Jan-Torsten Milde,et al.  Comparison of multimodal annotation tools: Workshop report , 2006 .

[10]  Peter Wittenburg,et al.  OntoELAN: an ontology-based linguistic multimedia annotator , 2004, IEEE Sixth International Symposium on Multimedia Software Engineering.

[11]  Marco Grassi.  Developing HEO Human Emotions Ontology , 2009, COST 2101/2102 Conference.

[12]  Kôiti Hasida,et al.  Towards an ISO Standard for Dialogue Act Annotation , 2010, LREC.

[13]  E. Mannens,et al.  XML to RDF Conversion: A Generic Approach , 2008, 2008 International Conference on Automated Solutions for Cross Media Content and Multi-Channel Distribution.

[14]  Steffen Staab,et al.  M-OntoMat-Annotizer: Image Annotation Linking Ontologies and Multimedia Low-Level Features , 2006, KES.

[15]  Michael Kipp,et al.  ANVIL - a generic annotation tool for multimodal dialogue , 2001, INTERSPEECH.

[16]  Stefan Kopp,et al.  Towards a Common Framework for Multimodal Generation: The Behavior Markup Language , 2006, IVA.