Ontology Evaluation through Usability Measures

Current ontology methodologies offer guidance towards knowledge acquisition, ontology development (design and conceptualization), formalization, evaluation, evolution and maintenance. Nevertheless, these methodologies describe most of expert involvements within ontology validation rather vaguely. The use of tailored usability methods for ontology evaluation could offer the establishment of certain quality measurements and aid the evaluation of modelling decisions, prior ontology implementation. This paper describes the experimental evaluation of a legal ontology, the Ontology of Professional Judicial Knowledge (OPJK), with the SUS questionnaire, a usability evaluation questionnaire tailored to ontology evaluation.

[1]  A. Gómez-Pérez,et al.  Evaluation of ontologies , 2001, Int. J. Intell. Syst..

[2]  Jonathan Earthy,et al.  The Benefits of Using ISO 13407: Human Centred Design Process for Interactive Systems , 2001, INTERACT.

[3]  Asunción Gómez-Pérez,et al.  Ontological Engineering: With Examples from the Areas of Knowledge Management, e-Commerce and the Semantic Web , 2004, Advanced Information and Knowledge Processing.

[4]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[5]  Nicola Guarino,et al.  Evaluating ontological decisions with OntoClean , 2002, CACM.

[6]  Pompeu Casanovas,et al.  Iuriservice: An Intelligent Frequently Asked Questions System to Assist Newly Appointed Judges , 2003, Law and the Semantic Web.

[7]  María Bárbara Álvarez Torres,et al.  On the Move to Meaningful Internet Systems 2004: OTM 2004 Workshops , 2004, Lecture Notes in Computer Science.

[8]  Mariano Fernández-López,et al.  Ontological Engineering , 2003, Encyclopedia of Database Systems.

[9]  Paola Velardi,et al.  TermExtractor: a Web Application to Learn the Common Terminology of Interest Groups and Research Communities , 2007 .

[10]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[11]  Núria Casellas Caralt.  Modelling legal knowledge through ontologies. OPJK: the ontology of professional judicial knowledge , 2008 .

[12]  Asunción Gómez-Pérez,et al.  Selection of Ontologies for the Semantic Web , 2003, ICWE.

[13]  Philipp Cimiano,et al.  Ontology Learning from Text: Methods, Evaluation and Applications , 2005 .

[14]  Gustavo Rossi,et al.  Web Engineering , 2001, Lecture Notes in Computer Science.

[15]  Pompeu Casanovas,et al.  An Ontology-Based Decision Support System for Judges , 2009, Law, Ontologies and the Semantic Web.

[16]  Amit P. Sheth,et al.  OntoQA: Metric-Based Ontology Quality Analysis , 2005 .

[17]  York Sure-Vetter,et al.  How to Design Better Ontology Metrics , 2007, ESWC.

[18]  Zahir Tari,et al.  On the Move to Meaningful Internet Systems 2005: OTM 2005 Workshops, OTM Confederated International Workshops and Posters, AWeSOMe, CAMS, GADA, MIOS+INTEROP, ORM, PhDS, SeBGIS, SWWS, and WOSE 2005, Agia Napa, Cyprus, October 31 - November 4, 2005, Proceedings , 2005, OTM Workshops.

[19]  Aldo Gangemi,et al.  Qood grid: A metaontology-based framework for ontology evaluation and selection , 2006, EON@WWW.

[20]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[21]  York Sure-Vetter,et al.  D1.2.3 Methods for ontology evaluation , 2004 .

[22]  Pompeu Casanovas,et al.  Law and the Semantic Web: Legal Ontologies, Methodologies, Legal Information Retrieval, and Applications , 2005, Law and the Semantic Web.

[23]  Michael Gruninger,et al.  Methodology for the Design and Evaluation of Ontologies , 1995, IJCAI 1995.

[24]  Pompeu Casanovas,et al.  OPJK into PROTON: Legal Domain Ontology Integration into an Upper-Level Ontology , 2005, OTM Workshops.

[25]  Pompeu Casanovas,et al.  Semantic enhancement for legal information retrieval: Iuriservice performance , 2007, ICAIL.

[26]  Lora Aroyo,et al.  The Semantic Web: Research and Applications , 2009, Lecture Notes in Computer Science.

[27]  Aldo Gangemi,et al.  Unit Tests for Ontologies , 2006, OTM Workshops.

[28]  Jakob Nielsen,et al.  How to Conduct a Heuristic Evaluation , 2006 .