A quantitative approach for analyzing the impact of tools on software productivity

Software tools are often adopted during large-scale software development in order to increase software productivity. Tools can affect the ability of the organization to deliver quality software systems; and they can affect software development cost and cycle-time. However, too frequently, the insertion of tools fails to yield the desired improvement. This constitutes a significant problem to software organizations because the adoption of a tool can incur considerable cost, and because the adoption of inappropriate tools can disrupt the development process. Predicting the impact of tool insertion has traditionally been difficult because one tool can have drastically different effects on different projects. Among the many factors that can affect the impact of tool insertion, two key factors are the development activities practiced in the process and the size of the adopting project. In order to improve the ability of software organizations to predict the impact of the insertion of tools on productivity, we have developed a method, called the Software Productivity Analysis Method (SPAM). The method helps in analyzing how the adoption of different technologies affects productivity in projects of different sizes when different development activities are used. To compare software productivity in different situations using SPAM, one must model the alternative activity sets, technologies and project sizes, using the specific modeling notation proposed here, and evaluate such models. A prototype tool has also been developed that supports the construction of such models and automates the analysis of software productivity in each case. To date, only a small number of methods have been proposed that facilitate the study of tool insertion impact. Arguably, SPAM is the first method that enables comparative analyses of software productivity with respect to technology, activities and project size. In order to validate the SPAM method it was used it in a case study in a large-scale industrial software development environment. The study revealed that there exists a complex interplay among the size of a software project, the technology and development activities employed in the project, and the productivity of software development. For instance, the study found that: (1) for a given tool, software productivity may peak at certain project sizes and drop down for both smaller and larger projects; (2) large projects with a relatively simple development process could lose productivity as a result of inserting a sophisticated tool; and (3) a software project that moves to a more rigorous process can regain much of the productivity losses entailed by inserting an appropriate tool. In the study, SPAM proved to be an effective vehicle for understanding the implications of tool insertion in a specific project. The method can help in making informed decisions on tool adoption by providing quantitative data on software productivity, development process, project size, and development technology.