Overall quality assessment of SKOS thesauri: An AHP-based approach

The article proposes a methodology for a thesauri quality assessment that supports decision-makers in selecting thesauri by exploiting an overall quality measure. This measure takes into account the subjective perceptions of the decision-maker according to the reuse of thesauri in a specific application context. The analytic hierarchy process methodology is adopted to capture both subjective and objective facets involved in the thesauri quality assessment, thus providing a ranking of the thesauri assessed. Our methodology is applied to a set of thesauri by using user-driven application contexts. A step-by-step explanation of how the approach supports the decision process in the creation, maintenance and exploitation of a framework of linked thesauri is provided.

[1]  Jens Lehmann,et al.  Assessing Linked Data Mappings Using Network Measures , 2012, ESWC.

[2]  Asunción Gómez-Pérez,et al.  A Maut aprroach for reusing domain ontologies on the basis of the NeOn Methodlogy , 2013 .

[3]  Antoine Isaac,et al.  Finding Quality Issues in SKOS Vocabularies , 2012, TPDL.

[4]  Ching-Lai Hwang,et al.  Multiple Attribute Decision Making: Methods and Applications - A State-of-the-Art Survey , 1981, Lecture Notes in Economics and Mathematical Systems.

[5]  Philipp Mayr,et al.  TheSoz: A SKOS representation of the thesaurus for the social sciences , 2012, Semantic Web.

[6]  Bernard Roy,et al.  Classement et choix en présence de points de vue multiples , 1968 .

[7]  Tom Heath,et al.  Linked Data: Evolving the Web into a Global Data Space , 2011, Linked Data.

[8]  Jens Lehmann,et al.  Quality assessment for Linked Data: A Survey , 2015, Semantic Web.

[9]  Asunción Gómez-Pérez,et al.  ONTOMETRIC: A Method to Choose the Appropriate Ontology , 2004, J. Database Manag..

[10]  Eero Hyvönen,et al.  Deploying National Ontology Services: From ONKI to Finto , 2014, International Semantic Web Conference.

[11]  Gail Hodge,et al.  Systems of Knowledge Organization for Digital Libraries: Beyond Traditional Authority Files , 2000 .

[12]  Kyung-Sun Kim,et al.  Selecting quality sources: Bridging the gap between the perception and use of information sources , 2011, J. Inf. Sci..

[13]  E. Triantaphyllou,et al.  A Sensitivity Analysis Approach for Some Deterministic Multi-Criteria Decision-Making Methods* , 1997 .

[14]  Evangelos Triantaphyllou,et al.  Multi-Criteria Decision Making: An Operations Research Approach , 1998 .

[15]  T. Saaty.  The Seven Pillars of the Analytic Hierarchy Process , 2001 .

[16]  Johannes Keizer,et al.  The AGROVOC Linked Dataset , 2013, Semantic Web.

[17]  Jyri Mustajoki,et al.  Comparison of Multi-Criteria Decision Analytical Software - Searching for ideas for developing a new EIA-specific multi-criteria software , 2013 .

[18]  Crawford Revie,et al.  Thesaurus-enhanced search interfaces , 2002, J. Inf. Sci..

[19]  Ching-Lai Hwang,et al.  Fuzzy Multiple Attribute Decision Making - Methods and Applications , 1992, Lecture Notes in Economics and Mathematical Systems.

[20]  Joseph Moses Juran,et al.  Quality-control handbook , 1951 .

[21]  Nicola Guarino,et al.  Evaluating ontological decisions with OntoClean , 2002, CACM.

[22]  T. Masuda.  Hierarchical sensitivity analysis of priority used in analytic hierarchy process , 1990 .

[23]  Ernest H. Forman,et al.  Decision by Objectives , 2001 .

[24]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[25]  Gianluca Demartini,et al.  Large-scale linked data integration using probabilistic reasoning and crowdsourcing , 2013, The VLDB Journal.

[26]  Andreas Abecker,et al.  Latest Developments of the Linked Thesaurus Framework for the Environment (LusTRE) , 2015, EnviroInfo/ICT4S.

[27]  T. Saaty.  Fundamentals of Decision Making and Priority Theory With the Analytic Hierarchy Process , 2000 .

[28]  A. Maurino,et al.  Quality Assessment for Linked Open Data : A Survey A Systematic Literature Review and Conceptual Framework , 2013 .

[29]  Christian Bizer,et al.  Quality-driven information filtering using the WIQA policy framework , 2009, J. Web Semant..

[30]  David Bawden,et al.  The dark side of information: overload, anxiety and other paradoxes and pathologies , 2009, J. Inf. Sci..

[31]  Eero Hyvönen,et al.  Improving the Quality of SKOS Vocabularies with Skosify , 2012, EKAW.

[32]  Edmundas Kazimieras Zavadskas,et al.  State of art surveys of overviews on MCDM/MADM methods , 2014 .

[33]  Asunción Gómez-Pérez,et al.  Assessing linkset quality for complementing third-party datasets , 2013, EDBT '13.

[34]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[35]  Giri Kumar Tayi,et al.  Examining data quality , 1998, CACM.

[36]  Bernard Roy,et al.  The Optimisation Problem Formulation: Criticism and Overstepping , 1981 .

[37]  Diane M. Strong,et al.  Beyond Accuracy: What Data Quality Means to Data Consumers , 1996, J. Manag. Inf. Syst..

[38]  Riccardo Albertoni,et al.  EARTh: An Environmental Application Reference Thesaurus in the Linked Open Data cloud , 2014, Semantic Web.

[39]  Thomas Redman,et al.  Data quality for the information age , 1996 .

[40]  Felix Naumann,et al.  Quality-Driven Query Answering for Integrated Information Systems , 2002, Lecture Notes in Computer Science.

[41]  Giovanni Bergamin,et al.  The Nuovo soggettario as a service for the linked data world , 2012 .

[42]  Simon K. Milton,et al.  Towards Quality Measures for Evaluating Thesauri , 2010, MTSR.

[43]  Alessio Ishizaka,et al.  AHPSort: an AHP-based method for sorting problems , 2012 .

[44]  Sean Bechhofer,et al.  SKOS Simple Knowledge Organization System Reference , 2009 .

[45]  Riccardo Albertoni,et al.  Environmental Thesauri under the Lens of Reusability , 2014, EGOVIS.

[46]  Bernhard Haslhofer,et al.  Using SKOS vocabularies for improving web search , 2013, WWW '13 Companion.

[47]  Sibs von Solms.  VALIDITY OF THE AHP/ANP: COMPARING APPLES AND ORANGES , 2011 .

[48]  Sushil Kumar,et al.  Analytic hierarchy process: An overview of applications , 2006, Eur. J. Oper. Res..

[49]  Gloria Bordogna,et al.  A linguistic decision making approach to assess the quality of volunteer geographic information for citizen science , 2014, Inf. Sci..

[50]  Asunción Gómez-Pérez,et al.  A MAUT Approach for Reusing Domain Ontologies on the Basis of the NeOn Methodology , 2013, Int. J. Inf. Technol. Decis. Mak..

[51]  Paolo Papotti,et al.  Introduction to the special issue on data quality , 2013, Inf. Syst..

[52]  Luis G. Vargas,et al.  Comparison of eigenvalue, logarithmic least squares and least squares methods in estimating ratios , 1984 .

[53]  Ali Selamat,et al.  Effect of thesaurus size on schema matching quality , 2014, Knowl. Based Syst..

[54]  Edie Rasmussen,et al.  Proceedings of the Second international conference on Theory and Practice of Digital Libraries , 2012 .

[55]  Tom DeMarco,et al.  Controlling Software Projects: Management, Measurement, and Estimates , 1986 .

[56]  Tom DeMarco,et al.  Controlling software projects : management, measurement & estimation , 1982 .

[57]  T. L. Saaty.  A Scaling Method for Priorities in Hierarchical Structures , 1977 .

[58]  Richard Y. Wang,et al.  Data quality assessment , 2002, CACM.

[59]  Osma Suominen,et al.  Assessing and Improving the Quality of SKOS Vocabularies , 2014, Journal on Data Semantics.

[60]  Riccardo Albertoni,et al.  A Linkset Quality Metric Measuring Multilingual Gain in SKOS Thesauri , 2015, LDQ@ESWC.

[61]  Christoph Lange,et al.  Luzzu -- A Framework for Linked Data Quality Assessment , 2016, 2016 IEEE Tenth International Conference on Semantic Computing (ICSC).

[62]  Thomas L. Saaty,et al.  Reflections and Projections on Creativity in Operations Research and Management Science: A Pressing Need for a Shift in Paradigm , 1998, Oper. Res..

[63]  Riccardo Albertoni,et al.  A multilingual/multicultural semantic-based approach to improve Data Sharing in a SDI for Nature Conservation , 2011, Int. J. Spatial Data Infrastructures Res..

[64]  Nikos Palavitsinis,et al.  A survey of knowledge organization systems in environmental sciences , 2009, ITEE.

[65]  Carlo Batini,et al.  Methodologies for data quality assessment and improvement , 2009, CSUR.

[66]  Hong-Gee Kim,et al.  A FCA-Based Ontology Construction for the Design of Class Hierarchy , 2005, ICCSA.

[67]  Jens Lehmann,et al.  TripleCheckMate: A Tool for Crowdsourcing the Quality Assessment of Linked Data , 2013, KESW.