On the impact of crosscutting concern projection on code measurement

Many concern metrics have been defined to quantify properties of crosscutting concerns, such as scattering, tangling, and dedication. To quantify these properties, concern metrics directly rely on the projection (assignment) of concerns into source code. Although concern identification tools have emerged over the last years, they are still rarely used in practice to support concern projection and, therefore, it is a task often performed manually. This means that the results of concern metrics are likely to be influenced by how accurately programmers assign concerns to code elements. Even though concern assignment is an important and long-standing problem in software engineering, its impact on accurate measures of crosscutting concerns has never been studied and quantified. This paper presents a series of 5 controlled experiments to quantify and analyse the impact of concern projection on crosscutting concern measures. A set of 80 participants from 4 different institutions projected 10 concern instances into the source code of two software systems. We analyse the accuracy of concern projections independently made by developers, and their impact on a set of 12 concern metrics. Our results suggest that: (i) programmers are conservative when projecting crosscutting concerns, (ii) all concern metrics suffer with such conservative behaviour, and (iii) fine-grained tangling measures are more sensitive to different concern projections than coarse-grained scattering metrics.