A FRAMEWORK FOR MEASURING QUALITY OF MODELS : EXPERIENCES FROM A SERIES OF CONTROLLED EXPERIMENTS

Tao Yue, Shaukat Ali, Maged Elaasar Certus Software V&V Center, Simula Research Laboratory, Oslo, Norway {tao, shaukat}@simula.no IBM Canada Ltd, Rational Software, Ottawa Lab, Canada melaasar@ca.ibm.com Abstract Controlled experiments in model-based software engineering, especially those involving human subjects performing modeling tasks, often require comparing models produced by experiment subjects with reference models, which are considered to be correct and complete. The purpose of such comparison is to assess the quality of models produced by experiment subjects so that experiment hypotheses can be accepted or rejected. The quality of models is typically measured quantitatively based on metrics. Manually defining such metrics for large modeling languages is often cumbersome and error-prone. It can also result in metrics that do not systematically consider relevant details and in turn may produce biased results.