Using Empirical Studies to Assess Software Development Approaches and Measurement Programs

Incremental development, software reuse, product families and component-based development seem to be the potent technologies to achieve benefits in productivity, quality and maintainability, and to reduce the risks of changes. These approaches have multiple and crosscutting impacts on development practices and quality attributes. Empirical studies in industry answer questions about why and when certain approaches are chosen, how these are applied with impact on single instances and how to generalize over classes or systems. Large, long-lived systems place more demands on software engineering approaches. Complexity is increased, systems should have the correct subset of functionality and be maintainable for several years to return the investment.The research in this thesis is based on several empirical studies performed at Ericsson in Grimstad, Norway and in the context of the Norwegian INCO project (INcremental and COmponent-Based Software Development). A product family with two large-scale products that have been developed incrementally is described. The work aimed to assess the impact of development approaches on quality and improve the practice in some aspects. The research has been a mixed-method design and the studies use qualitative data collected from sources such as web pages, text documents and own studies, as well as quantitative data from company’s data repositories for several releases of one product. The thesis contains five main novel contributions:C1. Empirical verification of reuse benefits. Quantitative analyses of defect reports, change requests and component size showed reuse benefits in terms of lower defect-density, higher stability between releases, and no significant difference in change-proneness between reused and non-reused components.C2. Increased understanding of the origin and type of changes in requirements in each release and changes of software between releases. A quantitative analysis of change requests showed that most changes are initiated by the organization. Perfective changes to functionality and quality attributes are most common. Functionality is enhanced and improved in each release, while quality attributes are mostly improved and have fewer changes in form of new requirements.C3. Developing an effort estimation method using use case specifications and the distribution of effort in different phases of incremental software development. The estimation method is tailored for complex use case specifications, incremental changes in these and reuse of software from previous releases. Historical data on effort spent in two releases are used to calibrate and validate the method.C4. Identifying metrics for a combination of reuse of software components and incremental development. Results of quantitative and qualitative studies are used to relate quality attributes to development practices and approaches, and to identify metrics for a combination of software reuse and incremental development.C5. Developing a data mining method for exploring industrial data repositories based on experience from the quantitative studies. This thesis also proposes how to improve the software processes for incremental development of product families. These are considered minor contributions:C6a. Adaptation of the Rational Unified Process for reuse to improve consistency between practice and the software process model.C6b. Improving techniques for incremental inspection of UML models to improve the quality of components. A controlled industrial experiment is performed.