Multimedia Tools and Applications Manuscript No. a Context-aware Middleware for Real-time Semantic Enrichment of Distributed Multimedia Metadata

This paper investigates the problem of the real-time integration and processing of multimedia metadata collected by a distributed sensor network. The discussed practical problem is the efficiency of the technologies used in creating a Knowledge Base in real-time. Specifically, an approach is proposed for the real-time, rule-based semantic enrichment of lower level context features with higher-level semantics. The distinguishing characteristic is the provision of an intelligent middleware-based architecture on which low level components such as sensors, feature extraction algorithms, data sources, and high level components such as application-specific ontologies can be plugged. Throughout the paper, Priamos, a middleware architecture based on Semantic Web technologies is presented, together with a stress-test of the system’s operation under two test case scenarios: A smart security surveillance application and a smart meeting room application. Performance measurements are conducted and corresponding results are exposed.

[1]  Chang-Hsing Lee,et al.  Scene-based event detection for baseball videos , 2007, J. Vis. Commun. Image Represent..

[2]  Olena Kaykova,et al.  Enabling Interoperability for Industrial Web Resources , 2005 .

[3]  Werner Nutt,et al.  Basic Description Logics , 2003, Description Logic Handbook.

[4]  J. Carroll,et al.  Jena: implementing the semantic web recommendations , 2004, WWW Alt. '04.

[5]  Diego López-de-Ipiña,et al.  SOAM: An Environment Adaptation Model for the Pervasive Semantic Web , 2006, ICCSA.

[6]  Siegfried Handschuh,et al.  Semantic annotation for knowledge management: Requirements and a survey of the state of the art , 2006, J. Web Semant..

[7]  François Goasdoué,et al.  Modeling Information Sources for Information Integration , 1999, EKAW.

[8]  Andy Seaborne,et al.  SPARQL/Update: A language for updating RDF graphs , 2007 .

[9]  Ghassan O. Karame,et al.  2D and 3D face localization for complex scenes , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[10]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[11]  Jeff Z. Pan,et al.  Towards Populating and Querying the Semantic Web , 2006 .

[12]  J. Soldatos,et al.  An ontology-based framework for dynamic resource management in ubiquitous computing environments , 2005, Second International Conference on Embedded Software and Systems (ICESS'05).

[13]  E. Prud hommeaux,et al.  SPARQL query language for RDF , 2011 .

[14]  Jeff Z. Pan,et al.  Multimedia annotations on the semantic Web , 2006, IEEE Multimedia.

[15]  Nikolas Mitrou,et al.  Priamos: a middleware architecture for real-time semantic annotation of context features , 2007 .

[16]  Dorin Comaniciu,et al.  Mean shift analysis and applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  Edward R. Dougherty,et al.  What is Real-Time Processing? , 1994 .

[18]  Jane Hunter,et al.  A Synchronous Multimedia Annotation System for Secure Collaboratories , 2006, 2006 Second IEEE International Conference on e-Science and Grid Computing (e-Science'06).

[19]  Lalana Kagal,et al.  A Semantic Context-Aware Access Control Framework for Secure Collaborations in Pervasive Computing Environments , 2006, SEMWEB.

[20]  Edward R. Dougherty,et al.  Introduction to Real-Time Imaging , 1995 .

[21]  Klara Nahrstedt,et al.  Gaia: a middleware platform for active spaces , 2002, MOCO.

[22]  Yarden Katz,et al.  Pellet: A practical OWL-DL reasoner , 2007, J. Web Semant..

[23]  Alexandra Poulovassilis,et al.  Event-Condition-Action Rule Languages for the Semantic Web , 2006, EDBT Workshops.

[24]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[25]  François Brémond,et al.  Automatic Video Interpretation: A Novel Algorithm for Temporal Scenario Recognition , 2003, IJCAI.

[26]  Ian Horrocks,et al.  From SHIQ and RDF to OWL: the making of a Web Ontology Language , 2003, J. Web Semant..

[27]  David Taniar,et al.  Computational Science and Its Applications - ICCSA 2006, International Conference, Glasgow, UK, May 8-11, 2006, Proceedings, Part I , 2006, ICCSA.

[28]  Ian Horrocks,et al.  Using an Expressive Description Logic: FaCT or Fiction? , 1998, KR.

[29]  Wonyong Sung,et al.  A statistical model-based voice activity detection , 1999, IEEE Signal Processing Letters.

[30]  Bijan Parsia,et al.  Task Computing - The Semantic Web Meets Pervasive Computing , 2003, SEMWEB.

[31]  Jorge S. Cardoso The Semantic Web Vision: Where Are We? , 2007, IEEE Intelligent Systems.

[32]  José Júlio Alferes,et al.  Active Rules in the Semantic Web: Dealing with Language Heterogeneity , 2005, RuleML.

[33]  Gerd Wagner,et al.  Rules and Rule Markup Languages for the Semantic Web , 2003, Lecture Notes in Computer Science.

[34]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[35]  N. Mitrou,et al.  A Semantic-Based Architecture for Sensor Data Fusion , 2008, 2008 The Second International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies.

[36]  Ora Lassila,et al.  Contextualizing applications via semantic middleware , 2005, The Second Annual International Conference on Mobile and Ubiquitous Systems: Networking and Services.

[37]  Paul Buitelaar,et al.  A Protégé Plug-In for Ontology Extraction from Text Based on Linguistic Analysis , 2004, ESWS.

[38]  Enrico Motta,et al.  Integrating Information Extraction, Ontology Learning and Semantic Browsing into Organizational Knowledge Processes , 2004, LSTKM@EKAW.

[39]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[40]  Doug Downey,et al.  Unsupervised named-entity extraction from the Web: An experimental study , 2005, Artif. Intell..

[41]  Min Xu,et al.  Multimodal Semantic Analysis and Annotation for Basketball Video , 2006, EURASIP J. Adv. Signal Process..

[42]  Shih-Fu Chang,et al.  Event detection in baseball video using superimposed caption recognition , 2002, MULTIMEDIA '02.

[43]  Boris Motik,et al.  A Comparison of Reasoning Techniques for Querying Large Description Logic ABoxes , 2006, LPAR.

[44]  Harry Chen,et al.  Semantic Web in the context broker architecture , 2004, Second IEEE Annual Conference on Pervasive Computing and Communications, 2004. Proceedings of the.

[45]  Diego Calvanese,et al.  The Description Logic Handbook , 2007 .

[46]  Steffen Staab,et al.  M-OntoMat-Annotizer: Image Annotation Linking Ontologies and Multimedia Low-Level Features , 2006, KES.

[47]  Ramakant Nevatia,et al.  VERL: An Ontology Framework for Representing and Annotating Video Events , 2005, IEEE Multim..

[48]  Gregory D. Abowd,et al.  A Context-Based Infrastructure for Smart Environments , 2000 .

[49]  Timothy W. Finin,et al.  A policy language for a pervasive computing environment , 2003, Proceedings POLICY 2003. IEEE 4th International Workshop on Policies for Distributed Systems and Networks.

[50]  Fabio Ciravegna,et al.  Cross-media document annotation and enrichment , 2006, SAAW@ISWC.

[51]  Aristodemos Pnevmatikakis,et al.  The AIT Multimodal Person Identification System for CLEAR 2007 , 2007, CLEAR.

[52]  L. Stein,et al.  OWL Web Ontology Language - Reference , 2004 .

[53]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[54]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[55]  Michael Sintek,et al.  Towards Bridging the Semantic Gap in Multimedia Annotation and Retrieval , 2006 .

[56]  Takeo Kanade,et al.  Computational Science and Its Applications - ICCSA 2009 , 2009 .