Spatiotemporal Reasoning for Complex Video Event Recognition in Content-Based Video Retrieval

Ontology-based representation of video scenes and events indicates a promising direction in content-based video retrieval. However, the multimedia ontologies described in the literature often lack formal grounding, and none of them are suitable for representing complex video scenes. This issue can be partially addressed using SWRL rules, which, however, can lead to undecidability. This paper presents a hybrid description logic-based architecture that employs general, spatial, temporal, and fuzzy axioms for video scene representation and automated reasoning-based scene interpretation, while achieving a favorable tradeoff between expressivity and reasoning complexity.

[1]  Carsten Lutz,et al.  A Correspondence between Temporal Description Logics , 2004, J. Appl. Non Class. Logics.

[2]  Euripides G. M. Petrakis,et al.  Temporal representation and reasoning in OWL 2 , 2017, Semantic Web.

[3]  Franz Baader,et al.  Runtime verification using the temporal description logic ALC-LTL revisited , 2014, J. Appl. Log..

[4]  Leslie F. Sikos.  A Novel Approach to Multimedia Ontology Engineering for Automated Reasoning over Audiovisual LOD Datasets , 2016, ACIIDS.

[5]  Enrico Franconi,et al.  A Temporal Description Logic for Reasoning about Actions and Plans , 1998, J. Artif. Intell. Res..

[6]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[7]  Carsten Lutz.  Interval-based Temporal Reasoning with General TBoxes , 2001, IJCAI.

[8]  Morteza Yousef Sanati.  A Metric Interval-based Temporal Description Logic , 2015 .

[9]  Remi Ronfard.  Shot-level description and matching of video content , 1997, Other Conferences.

[10]  Leslie F. Sikos.  Mastering Structured Data on the Semantic Web: From HTML5 Microdata to Linked Open Data , 2015 .

[11]  Heiner Stuckenschmidt,et al.  Towards Activity Recognition Using Probabilistic Description Logics , 2012, AAAI 2012.

[12]  Alessandro Artale,et al.  Tailoring Temporal Description Logics for Reasoning over Temporal Conceptual Models , 2011, FroCoS.

[13]  Kun Hu,et al.  The Temporal Description Logic TL-SI and its Decidability Algorithm , 2010, 2010 International Conference on Computational Aspects of Social Networks.

[14]  Michael Fisher,et al.  Temporal Representation and Reasoning , 2008, Handbook of Knowledge Representation.

[15]  Umberto Straccia,et al.  Fuzzy Ontology Representation using OWL 2 , 2010, Int. J. Approx. Reason..

[16]  Weiming Zhang,et al.  Video Semantic Content Analysis Framework Based on Ontology Combined MPEG-7 , 2007, Adaptive Multimedia Retrieval.

[17]  Enrico Franconi,et al.  A Temporal Description Logic for Reasoning over Conceptual Schemas and Queries , 2002, JELIA.

[18]  Miguel A. Patricio,et al.  Ontology-based context representation and reasoning for object tracking and scene interpretation in video , 2011, Expert Syst. Appl..

[19]  Songyang Lao,et al.  Video Semantic Content Analysis based on Ontology , 2007, International Machine Vision and Image Processing Conference (IMVIP 2007).

[20]  Leslie F. Sikos,et al.  3D model indexing in videos for content-based retrieval via X3D-based semantic enrichment and automated reasoning , 2017, Web3D.

[21]  Nacéra Bennacer,et al.  Semantic Mappings in Description Logics for Spatio-temporal Database Schema Integration , 2005, J. Data Semant..

[22]  Jean Christoph Jung,et al.  Lightweight Temporal Description Logics with Rigid Roles and Restricted TBoxes , 2015, IJCAI.

[23]  Alessandro Artale,et al.  A Cookbook for Temporal Conceptual Data Modelling with Description Logics , 2012, TOCL.

[24]  Adel M. Alimi,et al.  A fuzzy ontology: based framework for reasoning in visual video content analysis and indexing , 2011, MDMKDD '11.

[25]  Ioan Marius Bilasco,et al.  Events Detection Using a Video-Surveillance Ontology and a Rule-Based Approach , 2014, ECCV Workshops.

[26]  Norihiro Kamide,et al.  A Compatible Approach to Temporal Description Logics , 2010, Description Logics.

[27]  Volker Haarslev,et al.  Adding ternary complex roles to ALCRP(D) , 2002, Description Logics.

[28]  Leslie F. Sikos,et al.  RDF-powered semantic video annotation tools with concept mapping to Linked Data for next-generation video indexing: a comprehensive review , 2017, Multimedia Tools and Applications.

[29]  Zongtian Liu,et al.  A Temporal Description Logic for Reasoning about Action in Event , 2012 .

[30]  Volker Haarslev,et al.  A Logic-based Formalism for Reasoning about Visual Representations , 1999, J. Vis. Lang. Comput..

[31]  Salima Benbernou,et al.  A logic based approach for the multimedia data representation and retrieval , 2005, Seventh IEEE International Symposium on Multimedia (ISM'05).

[32]  Zhongzhi Shi,et al.  A dynamic description logic based system for video event detection , 2010 .

[33]  J.J. Leonard,et al.  Challenges for Autonomous Mobile Robots , 2007, International Machine Vision and Image Processing Conference (IMVIP 2007).

[34]  Diego Calvanese,et al.  Reasoning over Extended ER Models , 2007, ER.

[35]  Leslie F. Sikos.  Ontology-Based Structured Video Annotation for Content-Based Video Retrieval via Spatiotemporal Reasoning , 2018 .

[36]  Bernd Neumann,et al.  High-Level Expectations for Low-Level Image Processing , 2008, KI.

[37]  Leslie F. Sikos.  Description Logics in Multimedia Reasoning , 2017, Springer International Publishing.

[38]  Matteo Cristani,et al.  Practical Issues of Description Logics for Spatial Reasoning , 2009, AAAI Spring Symposium: Benchmarking of Qualitative Spatial and Temporal Reasoning Systems.

[39]  Rolf Grütter,et al.  Towards Spatial Reasoning in the Semantic Web: A Hybrid Knowledge Representation System Architecture , 2007, AGILE Conf..

[40]  Doo-Kwon Baik,et al.  Multimedia information retrieval based on spatiotemporal relationships using description logics for the semantic web , 2006, Int. J. Intell. Syst..

[41]  Ersin Elbasi.  Fuzzy Logic-Based Scenario Recognition from Video Sequences , 2013 .

[42]  Uzay Kaymak,et al.  tOWL : A Temporal Web Ontology Language , 2011 .

[43]  Alessandro Artale,et al.  Interval Temporal Description Logics , 2015, Description Logics.

[44]  Liang Chang,et al.  Verification of Branch-Time Property Based on Dynamic Description Logic , 2014, Intelligent Information Processing.

[45]  David M. W. Powers,et al.  Knowledge-Driven Video Information Retrieval with LOD: From Semi-Structured to Structured Video Metadata , 2015, ESAIR@CIKM.

[46]  Juan Carlos SanMiguel,et al.  An Ontology for Event Detection and its Application in Surveillance Video , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[47]  Bernd Neumann,et al.  Ontology-Based Reasoning Techniques for Multimedia Interpretation and Retrieval , 2008 .

[48]  Jean-Christophe Nebel,et al.  Episodic Reasoning for Vision-Based Human Action Recognition , 2014, TheScientificWorldJournal.

[49]  François Pachet,et al.  Using Description Logics for Indexing Audiovisual Documents , 1998, Description Logics.

[50]  Leslie F. Sikos,et al.  Mastering Structured Data on the Semantic Web , 2015, Apress.