Evaluating software engineering methods and tool part 1: The evaluation context and evaluation methods

In the last five issues of SIGSOFT Notes, Shari Lawrence Pfleeger has discussed the use of formal experiments to evaluate software engineering methods and tools [1]. Shari's articles were based on work she performed for the U.K. DESMET project which aimed to develop a methodology for evaluating software engineering methods and tools.The DESMET project identified a number of useful evaluation methods in addition to formal experiments, and Shari asked me to continue this column by describing some of other methods. As a starting point, I will give an overview of the scope of the DESMET methodology in this article and describe the nine different evaluation methods DESMET identified. In the next few articles I will discuss criteria for selecting a specific method in particular circumstances. Later I will present the DESMET guidelines for performing quantitative case studies and feature analysis.