OntoScene, A Logic-Based Scene Interpreter: Implementation and Application in the Rock Art Domain

We present OntoScene, a framework aimed at understanding the semantics of visual scenes starting from the semantics of their elements and the spatial relations holding between them. OntoScene exploits ontologies for representing knowledge and Prolog for specifying the interpretation rules that domain experts may adopt, and for implementing the SceneInterpreter engine. Ontologies allow the designer to formalize the domain in a reusable way, and make the system modular and interoperable with existing multiagent systems, while Prolog provides a solid basis to define complex rules of interpretation in a way that can be affordable even for people with no background in Computational Logics. The domain selected for experimenting OntoScene is that of prehistoric rock art, which provides us with a fascinating and challenging testbed. Under consideration in Theory and Practice of Logic Programming (TPLP)

[1]  Volker Haarslev,et al.  Combining Spatial and Terminological Reasoning , 1994, KI.

[2]  Bart Lamiroy,et al.  Inductive Logic Programming for Symbol Recognition , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[3]  Karell Bertet,et al.  An ontology-based framework for the automated analysis and interpretation of comic books' images , 2017, Inf. Sci..

[4]  Viviana Mascardi,et al.  A Holonic Multi-Agent System for Sketch, Image and Text Interpretation in the Rock Art Domain , 2014 .

[5]  Beniamino Di Martino,et al.  A rule‐based procedure for automatic recognition of design patterns in UML diagrams , 2016, Softw. Pract. Exp..

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Céline Hudelot,et al.  Towards ontologies for image interpretation and annotation , 2011, 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI).

[8]  B. Moore,et al.  Auditory and visual scene analysis: an overview , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[9]  Genny Tortora,et al.  Relation grammars and their application to multi-dimensional languages , 1991, J. Vis. Lang. Comput..

[10]  Daniela Briola.  Agents and Ontologies for a Smart Management of Heterogeneous Data: The IndianaMas System , 2015, IDC.

[11]  Agostino Dovier,et al.  A Comparison of CLP(FD) and ASP Solutions to NP-Complete Problems , 2005, ICLP.

[12]  M. Bar,et al.  Scene Vision: Making Sense of What We See , 2014 .

[13]  Christian Gerber,et al.  Holonic multi-agent systems , 1999 .

[14]  J. Henderson,et al.  High-level scene perception. , 1999, Annual review of psychology.

[15]  E. Rolls.  High-level vision: Object recognition and visual cognition, Shimon Ullman. MIT Press, Bradford (1996), ISBN 0 262 21013 4 , 1997 .

[16]  Viviana Mascardi,et al.  Agent‐oriented and ontology‐driven digital libraries: the IndianaMAS experience , 2017, Softw. Pract. Exp..

[17]  Volker Haarslev,et al.  Visual Spatial Query Languages: A Semantics Using Description Logic , 2002, Diagrammatic Representation and Reasoning.

[18]  Michael Wooldridge,et al.  Intelligent agents: theory and practice The Knowledge Engineering Review , 1995 .

[19]  Danny De Schreye,et al.  Answer Set Planning , 1999 .

[20]  Camille Jullian.  C. Bicknell, A guide to the prehistoric Rock Engravings in the Italian Maritime Alps, 1913 , 1915 .

[21]  Volker Haarslev,et al.  A Logic-based Formalism for Reasoning about Visual Representations , 1999, J. Vis. Lang. Comput..

[22]  Randall Davis,et al.  LADDER, a sketching language for user interface developers , 2005, Comput. Graph..

[23]  Bernd Meyer,et al.  Pictures depicting pictures on the specification of visual languages by visual grammars , 1992, Proceedings IEEE Workshop on Visual Languages.

[24]  Deborah L. McGuinness,et al.  OWL Web ontology language overview , 2004 .

[25]  Leslie F. Sikos.  Description Logics in Multimedia Reasoning , 2017, Springer International Publishing.

[26]  Henry de Lumley,et al.  The rock carvings of the Chalcolithic and ancient Bronze age from the Mont Bego area. The cosmogonic myths of the early metallurgic settlers in the southern Alps , 2015 .

[27]  Agostino Poggi,et al.  Developing Multi-agent Systems with JADE , 2007, ATAL.

[28]  Luc De Raedt,et al.  A Relational Distance-based Framework for Hierarchical Image Understanding , 2012, ICPRAM.

[29]  Werner Nutt,et al.  Terminological Knowledge Representation: A Proposal for a Terminological Logic , 1991, Description Logics.

[30]  Peter J. Stuckey,et al.  Programming with Constraints: An Introduction , 1998 .

[31]  Viviana Mascardi,et al.  Ontology-Driven Processing and Management of Digital Rock Art Objects in IndianaMAS , 2014, EuroMed.

[32]  Michele Risi,et al.  Sketch Grammars: a formalism for describing and recognizing diagrammatic sketch languages , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[33]  Kim Marriott,et al.  A declarative specification and semantics for visual languages , 1991, J. Vis. Lang. Comput..

[34]  Steffen Staab,et al.  What Is an Ontology? , 2009, Handbook on Ontologies.

[35]  Louis Weitzman,et al.  Unification-based grammars and tabular parsing for graphical languages , 1991, J. Vis. Lang. Comput..

[36]  Refractor.  Vision , 2000, The Lancet.

[37]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[38]  Sanjiang Li,et al.  Region Connection Calculus: Its models and composition table , 2003, Artif. Intell..

[39]  David Stuart Robertson,et al.  A Visual Syntax for Logic and Logic Programming , 1998, J. Vis. Lang. Comput..

[40]  Anthony G. Cohn,et al.  Modelling Topological and Metrical Properties in Physical Processes , 1989, KR.

[41]  Viviana Mascardi,et al.  OntologyBeanGenerator 5.0: Extending Ontology Concepts with Methods and Exceptions , 2018, WOA.

[42]  Germain Forestier,et al.  An Evolutionary Approach for Ontology Driven Image Interpretation , 2008, EvoWorkshops.

[43]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[44]  Ernest Friedman Hill,et al.  Jess in Action: Java Rule-Based Systems , 2003 .

[45]  Michel Rueher,et al.  VLP: a visual logic programming language , 1991, J. Vis. Lang. Comput..

[46]  Lauri Karttunen,et al.  D-PATR: A Development Environment for Unification-Based Grammars , 1986, COLING.

[47]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[48]  Henry de Lumley,et al.  Les gravures rupestres du Chalcolithique et de l’âge du Bronze ancien de la région du mont Bego. Les mythes cosmogoniques des premiers peuples métallurgistes des Alpes méridionales , 2009 .

[49]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[50]  Donald E. Knuth,et al.  Dancing links , 2000, cs/0011047.