Evaluation of the OQuaRE framework for ontology quality

The increasing importance of ontologies has resulted in the development of a large number of ontologies in both coordinated and non-coordinated efforts. The number and complexity of such ontologies make hard to ontology and tool developers to select which ontologies to use and reuse. So far, there are no mechanism for making such decisions in an informed manner. Consequently, methods for evaluating ontology quality are required. OQuaRE is a method for ontology quality evaluation which adapts the SQuaRE standard for software product quality to ontologies. OQuaRE has been applied to identify the strengths and weaknesses of different ontologies but, so far, this framework has not been evaluated itself. Therefore, in this paper we present the evaluation of OQuaRE, performed by an international panel of experts in ontology engineering. The results include the positive and negative aspects of the current version of OQuaRE, the completeness and utility of the quality metrics included in OQuaRE and the comparison between the results of the manual evaluations done by the experts and the ones obtained by a software implementation of OQuaRE.

[1]  Jesualdo Tomás Fernández-Breis,et al.  Towards SQUaRE based Quality Evaluation Framework for Ontologies , 2010 .

[2]  Asunción Gómez-Pérez,et al.  Common Pitfalls in Ontology Development , 2009, CAEPIA.

[3]  Nicola Guarino,et al.  An Overview of OntoClean , 2004, Handbook on Ontologies.

[4]  Asunción Gómez-Pérez,et al.  ODEVAL: A Tool for Evaluating RDF(S), DAML+OIL and OWL Concept Taxonomies , 2004, AIAI.

[5]  Asunción Gómez-Pérez,et al.  Ontology Evaluation , 2004, Handbook on Ontologies.

[6]  Sallie M. Henry,et al.  Object-oriented metrics that predict maintainability , 1993, J. Syst. Softw..

[7]  Harith Alani,et al.  Ranking Ontologies with AKTiveRank , 2006, SEMWEB.

[8]  Gil Alterovitz,et al.  Knowledge-Based Bioinformatics: From analysis to interpretation , 2010 .

[9]  Nilesh Anand,et al.  GenCLOn: An ontology for city logistics , 2012, Expert Syst. Appl..

[10]  Robert Stevens,et al.  Application of Ontologies in Bioinformatics , 2009, Handbook on Ontologies.

[11]  Elena Beisswanger,et al.  BioTop: An upper domain ontology for the life sciencesA description of its current structure, contents and interfaces to OBO ontologies , 2008, Appl. Ontology.

[12]  Mutao Huang,et al.  Enhance discovery and retrieval of geospatial data using SOA and Semantic Web technologies , 2012, Expert Syst. Appl..

[13]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[14]  Asunción Gómez-Pérez,et al.  A Double Classification of Common Pitfalls in Ontologies , 2010 .

[15]  M. Ashburner,et al.  Gene Ontology: tool for the unification of biology , 2000, Nature Genetics.

[16]  Martin Hepp,et al.  GoodRelations: An Ontology for Describing Products and Services Offers on the Web , 2008, EKAW.

[17]  Hajo Rijgersberg,et al.  How semantics can improve engineering processes: A case of units of measure and quantities , 2011, Adv. Eng. Informatics.

[18]  Samir Tartir,et al.  Ontology Evaluation and Ranking using OntoQA , 2007 .

[19]  Jesualdo Tomás Fernández-Breis,et al.  A Quality Evaluation Framework for Bio-Ontologies , 2009 .

[20]  M. Petró‐Turza,et al.  The International Organization for Standardization. , 2003 .

[21]  Enrico Motta,et al.  Ontology Selection: Ontology Evaluation on the Real Semantic Web , 2006, EON@WWW.

[22]  Nathalie Aussenac-Gilles,et al.  OQuaRE: A SQuaRE-based Approach for Evaluating the Quality of Ontologies , 2011, J. Res. Pract. Inf. Technol..

[23]  Ismailcem Budak Arpinar,et al.  Ontology Evaluation and Ranking using OntoQA , 2007, International Conference on Semantic Computing (ICSC 2007).

[24]  Heon Young Yeom,et al.  A comprehensive framework for the evaluation of ontology modularization , 2012, Expert Syst. Appl..

[25]  Letha H. Etzkorn,et al.  Cohesion Metrics for Ontology Design and Application , 2005 .

[26]  Egon Berghout,et al.  The Goal/Question/Metric Method: , 2000 .

[27]  J E Rogers,et al.  Quality Assurance of Medical Ontologies , 2006, Methods of Information in Medicine.

[28]  M. Ashburner,et al.  An ontology for cell types , 2005, Genome Biology.

[29]  Derek H. Sleeman,et al.  CleanONTO: Evaluating Taxonomic Relationships in Ontologies , 2006, EON@WWW.

[30]  Besiki Stvilia,et al.  A model for ontology quality evaluation , 2007, First Monday.

[31]  Marko Grobelnik,et al.  A SURVEY OF ONTOLOGY EVALUATION TECHNIQUES , 2005 .

[32]  Asunción Gómez-Pérez,et al.  ONTOMETRIC: A Method to Choose the Appropriate Ontology , 2004, J. Database Manag..

[33]  Aldo Gangemi,et al.  Modelling Ontology Evaluation and Validation , 2006, ESWC.