A framework for evaluating and ranking ontologies

Reusing the increasing number of available ontologies is extremely important for the advancement and the growth of Semantic Web, because it reduces the required costs, efforts and amount of time to build new ontologies. However, the lack of standard frameworks to assess their quality and their suitability for being used in a specific context remains a barrier to their wide adoption and reuse. To address this problem, we present in this paper a framework for evaluating and ranking ontologies, with three distinctive features. First, it provides an advanced retrieval mechanism that gathers the best set of candidate ontologies. Second, it defines a rich set of metrics for assessing several ontology aspects. Third, it supplies a helpful way for visualising and interpreting the results. We implemented and evaluated the proposed metrics. The results of these evaluations are presented, indicating the usefulness of the overall framework.

[1]  Enrico Motta,et al.  Watson, more than a Semantic Web search engine , 2011, Semantic Web.

[2]  Wei Yu,et al.  Ranking Ontology Based on Structure Analysis , 2009, 2009 Second International Symposium on Knowledge Acquisition and Modeling.

[3]  Luciano Tarricone,et al.  EVALUATING SCIENTIFIC DOMAIN ONTOLOGIES FOR THE ELECTROMAGNETIC KNOWLEDGE DOMAIN : A GENERAL METHODOLOGY , 2011 .

[4]  Asunción Gómez-Pérez,et al.  Why Evaluate Ontology Technologies? Because It Works! , 2004, IEEE Intell. Syst..

[5]  Riichiro Mizoguchi,et al.  A Quality Assurance Framework for Ontology Construction and Refinement , 2011, AWIC.

[6]  Jeff Z. Pan,et al.  ONTOSEARCH2: SEARCHING AND QUERYING WEB ONTOLOGIES , 2007 .

[7]  Yinglong Ma,et al.  A Proposal for Stable Semantic Metrics Based on Evolving Ontologies , 2009, 2009 International Joint Conference on Artificial Intelligence.

[8]  Muhammad Abdul Qadir,et al.  Semantic Inconsistency Errors in Ontology , 2007 .

[9]  Jürgen Umbrich,et al.  SWSE: Answers Before Links! , 2007, Semantic Web Challenge.

[10]  Enrico Motta,et al.  Next Generation Semantic Web Applications , 2006, ASWC.

[11]  Mathieu d'Aquin,et al.  Cupboard - A Place to Expose Your Ontologies to Applications and the Community , 2009, ESWC.

[12]  Letha H. Etzkorn,et al.  Cohesion Metrics for Ontology Design and Application , 2005 .

[13]  Leo Obrst,et al.  The Evaluation of Ontologies , 2007 .

[14]  Rim Djedidi,et al.  ONTO-EVOAL an Ontology Evolution Approach Guided by Pattern Modeling and Quality Evaluation , 2010, FoIKS.

[15]  Elena García Barriocanal,et al.  Empirical findings on ontology metrics , 2012, Expert Syst. Appl..

[16]  Patrice Degoulet,et al.  Terminology extraction from text to build an ontology in surgical intensive care , 2002, AMIA.

[17]  Sadok Ben Yahia,et al.  A Novel Global Measure Approach based on Ontology Spectrum to Evaluate Ontology Enrichment , 2012 .

[18]  Colin Allen,et al.  Two Methods for Evaluating Dynamic Ontologies , 2010, KEOD.

[19]  Eyal Oren,et al.  Sindice.com: a document-oriented lookup index for open linked data , 2008, Int. J. Metadata Semant. Ontologies.

[20]  Asunción Gómez-Pérez,et al.  ONTOMETRIC: A Method to Choose the Appropriate Ontology , 2004, J. Database Manag..

[21]  Hongyu Zhang,et al.  Measuring design complexity of semantic web ontologies , 2010, J. Syst. Softw..

[22]  Harith Alani,et al.  Ranking Ontologies with AKTiveRank , 2006, SEMWEB.

[23]  Krys J. Kochut,et al.  BRAHMS: A WorkBench RDF Store and High Performance Memory System for Semantic Association Discovery , 2005, SEMWEB.

[24]  Lennart Martens,et al.  The Ontology Lookup Service: more data and better tools for controlled vocabulary queries , 2008, Nucleic Acids Res..

[25]  Timothy W. Finin,et al.  Swoogle: a search and metadata engine for the semantic web , 2004, CIKM '04.

[26]  Yugyung Lee,et al.  OntoKhoj: a semantic web portal for ontology searching, ranking and classification , 2003, WIDM '03.

[27]  Thomas R. Gruber,et al.  A translation approach to portable ontology specifications , 1993, Knowl. Acquis..

[28]  Yuzhong Qu,et al.  Searching Linked Objects with Falcons: Approach, Implementation and Evaluation , 2009, Int. J. Semantic Web Inf. Syst..

[29]  Christopher G. Chute,et al.  BioPortal: ontologies and integrated data resources at the click of a mouse , 2009, Nucleic Acids Res..

[30]  Nathalie Aussenac-Gilles,et al.  OQuaRE: A SQuaRE-based Approach for Evaluating the Quality of Ontologies , 2011, J. Res. Pract. Inf. Technol..

[31]  Yoav Goldberg,et al.  Ontology Evaluation through Text Classification , 2009, APWeb/WAIM Workshops.

[32]  Amit P. Sheth,et al.  OntoQA: Metric-Based Ontology Quality Analysis , 2005 .

[33]  Marko Grobelnik,et al.  A SURVEY OF ONTOLOGY EVALUATION TECHNIQUES , 2005 .

[34]  Samir Tartir,et al.  Ontology Evaluation and Ranking using OntoQA , 2007 .

[35]  M. Ashburner,et al.  The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration , 2007, Nature Biotechnology.