CONTENTUS - technologies for next generation multimedia libraries - Automatic multimedia processing for semantic search

An ever-growing amount of digitized content urges libraries and archives to integrate new media types from a large number of origins such as publishers, record labels and film archives, into their existing collections. This is a challenging task, since the multimedia content itself as well as the associated metadata is inherently heterogeneous—the different sources lead to different data structures, data quality and trustworthiness. This paper presents the contentus approach J. Nandzik (B) · N. Flores-Herr Acosta Consult GmbH, Zeißelstraße 15 HH, 60318 Frankfurt am Main, Germany e-mail: jn@acosta-consult.de N. Flores-Herr e-mail: nf@acosta-consult.de B. Litz · A. Löhden Deutsche Nationalbibliothek, Informationstechnik, Adickesallee 1, 60322 Frankfurt am Main, Germany B. Litz e-mail: b.litz@dnb.de A. Löhden e-mail: a.loehden@dnb.de I. Konya · D. Baum · A. Bergholz Fraunhofer IAIS, Schloss Birlinghoven, 53754 Sankt Augustin, Germany I. Konya e-mail: iuliu.vasile.konya@iais.fraunhofer.de D. Baum e-mail: doris.baum@iais.fraunhofer.de A. Bergholz e-mail: andre.bergholz@iais.fraunhofer.de D. Schönfuß mufin GmbH, Büro Dresden, August-Bebel-Straße 36, 01219 Dresden, Germany e-mail: dschoenfuss@mufin.com

[1]  Yaron Goland,et al.  Web Services Business Process Execution Language , 2009, Encyclopedia of Database Systems.

[2]  Joachim Köhler,et al.  Constrained Subword Units for Speaker Recognition , 2010, Odyssey.

[3]  Harald Sack,et al.  Exploratory Semantic Video Search with yovisto , 2010, 2010 IEEE Fourth International Conference on Semantic Computing.

[4]  Petasis George,et al.  Semi-automated ontology learning : the BOEMIE approach , 2009 .

[5]  Joachim Köhler,et al.  DiSCo - A German Evaluation Corpus for Challenging Problems in the Broadcast Domain , 2010, LREC.

[6]  Adrian Ulges,et al.  A System That Learns to Tag Videos by Watching Youtube , 2008, ICVS.

[7]  Hsin-Min Wang,et al.  BIC-Based Speaker Segmentation Using Divide-and-Conquer Strategies With Application to Speaker Diarization , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[8]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[9]  Yiannis Kompatsiaris,et al.  Semantic Multimedia and Ontologies: Theory and Applications , 2008 .

[10]  Ilaria Bartolini,et al.  Shiatsu: semantic-based hierarchical automatic tagging of videos by segmentation using cuts , 2010, AIEMPro '10.

[11]  Yiannis Kompatsiaris,et al.  Advances in semantic multimedia analysis for personalised content access , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[12]  Edoardo Greppi.  FAO (Food and Agriculture Organization of the United Nations) , 1981 .

[13]  Dietrich Schüller.  International Association of Sound and Audiovisual Archives , 2007 .

[14]  Yiannis Kompatsiaris,et al.  A Survey of Semantic Image and Video Annotation Tools , 2011, Knowledge-Driven Multimedia Information Extraction and Ontology Evolution.

[15]  Stefan Eickeler,et al.  A new quality assessment and improvement system for print media , 2012, EURASIP J. Adv. Signal Process..

[16]  a. hess.  CONTENTUS – Towards Semantic Multimedia Libraries , 2010 .

[17]  Thomas M. Breuel,et al.  High Performance Document Layout Analysis , 2003 .

[18]  Wolfgang Nejdl,et al.  PHAROS - Platform For Search of Audiovisual Resources Across Online Spaces , 2006, SAMT.

[19]  Chrisa Tsinaraki,et al.  An MPEG-7 query language and a user preference model that allow semantic retrieval and filtering of multimedia content , 2007, Multimedia Systems.

[20]  Rong Yan,et al.  A review of text and image retrieval approaches for broadcast news video , 2007, Information Retrieval.

[21]  Arnold W. M. Smeulders,et al.  Visual-Concept Search Solved? , 2010, Computer.

[22]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[23]  Basilios Gatos,et al.  Page Segmentation Competition , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[24]  Anil K. Jain,et al.  Document Representation and Its Application to Page Decomposition , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[26]  Ioannis Pratikakis,et al.  Automatic Table Detection in Document Images , 2005, ICAPR.

[27]  Carol Peters,et al.  The MultiMatch Prototype: Multilingual/Multimedia Search for Cultural Heritage Objects , 2008, ECDL.

[28]  Ingeborg Sølvberg,et al.  Semantic Data Integration Framework in Peer-to-Peer based Digital Libraries , 2005, J. Digit. Inf. Manag..

[29]  Ugo Corda.  Multimedia Semantics from MPEG-7 Metadata to Semantic Web Ontologies , 2008 .

[30]  Huajun Chen,et al.  The Semantic Web , 2011, Lecture Notes in Computer Science.

[31]  Marcel Worring,et al.  Concept-Based Video Retrieval , 2009, Found. Trends Inf. Retr..

[32]  Steffen Staab,et al.  Semantic Multimedia: First International Conference on Semantic and Digital Media Technologies, SAMT 2006Athens, Greece, December 6-8, 2006Proceedings (Lecture Notes in Computer Science) , 2007 .

[33]  Jan Hannemann,et al.  Linked Data for Libraries , 2010 .

[34]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[35]  Nenghai Yu,et al.  Flickr distance , 2008, ACM Multimedia.

[36]  Patrick Ndjiki-Nya,et al.  Fully automatic inpainting method for complex image content , 2009, 2009 10th Workshop on Image Analysis for Multimedia Interactive Services.

[37]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[38]  Stefan Müller,et al.  Scratch detection supported by coherency analysis of motion vector fields , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[39]  Ian H. Witten,et al.  How to Build a Digital Library , 2002 .

[40]  George Buchanan,et al.  Semantics in Greenstone , 2009, Semantic Digital Libraries.

[41]  Christian Petersohn.  Fraunhofer HHI at TRECVID 2004: Shot Boundary Detection System , 2004, TRECVID.

[42]  Shih-Fu Chang,et al.  Enabling MPEG-7 structural and semantic descriptions in retrieval applications , 2007, J. Assoc. Inf. Sci. Technol..

[43]  Marios C. Angelides,et al.  From MPEG-7 user interaction tools to hanging basket models: bridging the gap , 2009, Multimedia Tools and Applications.

[44]  More than the Sum of its Parts : CONTENTUS – A Semantic Multimodal Search User Interface , 2010 .

[45]  Ramesh A. Gopinath,et al.  Improved speaker segmentation and segments clustering using the bayesian information criterion , 1999, EUROSPEECH.

[46]  Christoph Seibert,et al.  Constant-Time Locally Optimal Adaptive Binarization , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[47]  Marcel Worring,et al.  Semantic Image and Video Indexing in Broad Domains , 2007, IEEE Trans. Multim..

[48]  Christian Petersohn,et al.  Temporal video structuring for preservation and annotation of video content , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[49]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[50]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[51]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[52]  Doris Baum.  Topic-based speaker recognition for German parliamentary speeches , 2009, 2009 IEEE Workshop on Automatic Speech Recognition & Understanding.

[53]  Chrisa Tsinaraki,et al.  MPEG-7 and the Semantic Web , 2007 .

[54]  Lina J. Karam,et al.  A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) , 2009, IEEE Transactions on Image Processing.