Comprehensibility of orthogonal variability modeling languages: the cases of CVL and OVM

As the complexity and variety of systems and software products have increased, the ability to manage their variability effectively and efficiently became crucial. To this end, variability can be specified either as an integral part of the development artifacts or in a separate orthogonal variability model. Lately, orthogonal variability models attract a lot of attention due to the fact that they do not require changing the complexity of the development artifacts and can be used in conjunction with different development artifacts. Despite this attention and to the best of our knowledge, no empirical study examined the comprehensibility of orthogonal variability models. In this work, we conducted an exploratory experiment to examine potential comprehension problems in two common orthogonal variability modeling languages, namely, Common Variability Language (CVL) and Orthogonal Variability Model (OVM). We examined the comprehensibility of the variability models and their relations to the development artifacts for novice users. To measure comprehensibility we used comprehension score (i.e., percentage of correct solution), time spent to complete tasks, and participants' perception of difficulty of different model constructs. The results showed high comprehensibility of the variability models, but low comprehensibility of the relations between the variability models and the development artifacts. Although the comprehensibility of CVL and OVM was similar in terms of comprehension score and time spent to complete tasks, novice users perceived OVM as more difficult to comprehend.