Evaluating scalable matching tools: A quality-oriented approach

Actually, the evaluation of matching tools is an entire, complex and complicated research subject which we are interested in. Complex because matching systems can regroup several matching techniques and complicated considering their multiple users. Considering quality as an important element to define, use and evolve particular systems (as information and manufacturing systems), we extend traditional approaches and we propose an evaluation approach based on software product quality principles. In this paper, we offer an evaluation method based on a quality model (characteristics, sub-characteristics, measures...) adapted to the specificities of scalable matching tools. To illustrate our approach, we provide some evaluation results over two scalable matching tools COMA++ and PLASMA.

[1]  Aïcha-Nabila Benharkat,et al.  Méthodologie de Matching à large Echelle de schémas XML , 2009, INFORSID.

[2]  Zohra Bellahsene,et al.  On Evaluating Schema Matching and Mapping , 2011, Schema Matching and Mapping.

[3]  Yuzhong Qu,et al.  FalconAO: Aligning Ontologies with Falcon , 2005, Integrating Ontologies.

[4]  Fausto Giunchiglia,et al.  A Large Scale Dataset for the Evaluation of Matching Systems , 2006 .

[5]  Zohra Bellahsene,et al.  XBenchMatch: a Benchmark for XML Schema Matching Tools , 2007, VLDB.

[6]  Jérôme Euzenat,et al.  Semantic Precision and Recall for Ontology Alignment Evaluation , 2007, IJCAI.

[7]  Mario Piattini,et al.  Towards a consistent terminology for software measurement , 2006, Inf. Softw. Technol..

[8]  Aïcha-Nabila Benharkat,et al.  Benchmarking XML-Schema Matching Algorithms for Improving Automated Tuning , 2007, 2007 IEEE/ACS International Conference on Computer Systems and Applications.

[9]  Connie U. Smith,et al.  QSEM: Quantitative Scalability Evaluation Method , 2005, Int. CMG Conference.

[10]  J. Euzenat,et al.  Ontology Matching , 2007, Springer Berlin Heidelberg.

[11]  Jérôme Euzenat,et al.  Ten Challenges for Ontology Matching , 2008, OTM Conferences.

[12]  Erhard Rahm,et al.  Schema and ontology matching with COMA++ , 2005, SIGMOD '05.

[13]  Philip A. Bernstein,et al.  Industrial-strength schema matching , 2004, SGMD.

[14]  Erhard Rahm,et al.  Matching large schemas: Approaches and evaluation , 2007, Inf. Syst..

[15]  Aïcha-Nabila Benharkat,et al.  Towards a More Scalable Schema Matching: A Novel Approach , 2010, Int. J. Distributed Syst. Technol..

[16]  Norman F. Schneidewind,et al.  IEEE Standard For A Software Quality Metrics Methodology Revision And Reaffirmation , 1997, Proceedings of IEEE International Symposium on Software Engineering Standards.

[17]  Aïcha-Nabila Benharkat,et al.  Improving Real World Schema Matching with Decomposition Process , 2010, ICEIS.

[18]  Gunter Saake,et al.  Combining Effectiveness and Efficiency for Schema Matching Evaluation , 2008, MBSDI.

[19]  B. Kitchenham,et al.  DESMET : A method for evaluating Software Engineering methods and tools , 2000 .

[20]  Erhard Rahm,et al.  Recent Advances in Schema and Ontology Evolution , 2011, Schema Matching and Mapping.

[21]  Mikalai Yatskevich,et al.  Preliminary Evaluation of Schema Matching Systems , 2003 .

[22]  Erhard Rahm,et al.  Comparison of Schema Matching Evaluations , 2002, Web, Web-Services, and Database Systems.

[23]  Kajal T. Claypool,et al.  QoM: Qualitative and Quantitative Schema Match Measure , 2003, ER.

[24]  Erhard Rahm,et al.  Similarity flooding: a versatile graph matching algorithm and its application to schema matching , 2002, Proceedings 18th International Conference on Data Engineering.

[25]  Erhard Rahm,et al.  A survey of approaches to automatic schema matching , 2001, The VLDB Journal.

[26]  G. Steiner,et al.  A CHINESE PROVERB , 2013 .