Empirical evaluation of issue based variability modeling using the experimental survey technique

Though surveys are effective in data collection, they are rarely used as a technique for empirical evaluation in software engineering. Experimental surveys (surveys with experimental design) are frequently used in social and medical sciences, but are not exploited adequately in software engineering. This paper does an attempt to use the experimental survey technique for the empirical evaluation of a new product line requirements engineering approach called issue-based variability modeling. The focus of this paper is mainly on survey and questionnaire design, and covers issues such as sampling and statistical analysis. In this survey, a population of 34 professionals was sampled into an experimental and a control group using stratified random sampling. A workshop was organized with the experimental group using issue-based variability modeling while the control group used an existing variability modeling approach. At the end of the workshop a self-administrated questionnaire was given to both the groups and the data was collected. The survey data analysis was done using basic statistics.