A Snapshot of Ontology Evaluation Criteria and Strategies

Ontologies are key to information retrieval, semantic integration of datasets, and semantic similarity analyses. Evaluating ontologies (especially defining what constitutes a "good" or "better" ontology) is therefore of central importance for the Semantic Web community. Various criteria have been introduced in the literature to evaluate ontologies, and this article classifies them according to their relevance to the design or the implementation phase of ontology development. In addition, the article compiles strategies for ontology evaluation based on ontologies published until 2017 in two outlets: the Semantic Web Journal, and the Journal of Web Semantics. Gaps and opportunities for future research on ontology evaluation are exposed towards the end of the paper.

[1]  Marcela Vegetti,et al.  Towards ontology evaluation across the life cycleThe Ontology Summit 2013 , 2013, Appl. Ontology.

[2]  Nicola Guarino,et al.  Evaluating ontological decisions with OntoClean , 2002, CACM.

[3]  N. L. Wilson.  Substances Without Substrata , 1959 .

[4]  Nicola Guarino,et al.  Ontologies and Knowledge Bases. Towards a Terminological Clarification , 1995 .

[5]  Nicola Guarino,et al.  Formal Ontology and Information Systems , 1998 .

[6]  Werner Kuhn.  Modeling vs encoding for the Semantic Web , 2010, Semantic Web.

[7]  Steffen Staab,et al.  Ontology Engineering Methodology , 2009, Handbook on Ontologies.

[8]  James A. Thom,et al.  Requirements-oriented methodology for evaluating ontologies , 2009, Inf. Syst..

[9]  Giancarlo Guizzardi,et al.  On Ontology, ontologies, Conceptualizations, Modeling Languages, and (Meta)Models , 2007, DB&IS.

[10]  Thomas R. Gruber,et al.  Toward principles for the design of ontologies used for knowledge sharing? , 1995, Int. J. Hum. Comput. Stud..

[11]  Barry Smith,et al.  A spatio-temporal ontology for geographic information integration , 2009, Int. J. Geogr. Inf. Sci..

[12]  Carole D. Hafner,et al.  The State of the Art in Ontology Design: A Survey and Comparative Review , 1997, AI Mag..

[13]  Nicola Guarino,et al.  Applied ontology: The next decade begins , 2015, Appl. Ontology.

[14]  Dimitrios Tzovaras,et al.  A Methodological Approach for Ontology Evaluation and Refinement , 2008 .

[15]  Asunción Gómez-Pérez,et al.  ONTOMETRIC: A Method to Choose the Appropriate Ontology , 2004, J. Database Manag..

[16]  Maureen Donnelly,et al.  Logical properties of foundational relations in bio-ontologies , 2007, Artif. Intell. Medicine.

[17]  Kurt Sandkuhl,et al.  The State of Ontology Pattern Research: A Systematic Review of ISWC, ESWC and ASWC 2005-2009 , 2010, WOP.

[18]  Aldo Gangemi,et al.  Modelling Ontology Evaluation and Validation , 2006, ESWC.

[19]  Krzysztof Janowicz,et al.  Thoughts on the complex relation between linked data, semantic annotations, and ontologies , 2013, ESAIR '13.

[20]  Giancarlo Guizzardi,et al.  Theoretical foundations and engineering tools for building ontologies as reference conceptual models , 2010, Semantic Web.

[21]  Aldo Gangemi,et al.  Foundations for service ontologies: aligning OWL-S to dolce , 2004, WWW '04.

[22]  Michael Uschold,et al.  Ontologies: principles, methods and applications , 1996, The Knowledge Engineering Review.

[23]  Florian Probst,et al.  DOLCE ROCKS: Integrating Geoscience Ontologies with DOLCE , 2008, AAAI Spring Symposium: Semantic Scientific Knowledge Integration.

[24]  Asunción Gómez-Pérez,et al.  Evaluation of ontologies , 2001, International Journal of Intelligent Systems.

[25]  Asunción Gómez-Pérez,et al.  Ontology Evaluation , 2004, Handbook on Ontologies.

[26]  Andrew U. Frank,et al.  Ontology for Spatio-temporal Databases , 2003, Spatio-Temporal Databases: The CHOROCHRONOS Approach.

[27]  Michel Dumontier,et al.  Towards quantitative measures in applied ontology , 2012, ArXiv.

[28]  Dunja Mladenic,et al.  Automatic Evaluation of Ontologies , 2007 .

[29]  Michael Gruninger,et al.  Methodology for the Design and Evaluation of Ontologies , 1995, IJCAI 1995.

[30]  Christoph Stasch,et al.  Meaningful spatial prediction and aggregation , 2014, Environ. Model. Softw..

[31]  T. Bittner.  An Ontology for Spatio-temporal Databases , 2001 .

[32]  J. Wobbrock,et al.  Research contributions in human-computer interaction , 2016, Interactions.

[33]  Michael Grüninger,et al.  Applied Ontology: A foreword by the new Editors-in-Chief , 2017, Appl. Ontology.

[34]  Marko Grobelnik,et al.  A SURVEY OF ONTOLOGY EVALUATION TECHNIQUES , 2005 .

[35]  Asunción Gómez-Pérez,et al.  Methodologies, tools and languages for building ontologies: Where is their meeting point? , 2003, Data Knowl. Eng..