Assessing effort estimation models for corrective maintenance through empirical studies

Abstract We present an empirical assessment and improvement of the effort estimation model for corrective maintenance adopted in a major international software enterprise. Our study was composed of two phases. In the first phase we used multiple linear regression analysis to construct effort estimation models validated against real data collected from five corrective maintenance projects. The model previously adopted by the subject company used as predictors the size of the system being maintained and the number of maintenance tasks. While this model was not linear, we show that a linear model including the same variables achieved better performances. Also we show that greater improvements in the model performances can be achieved if the types of the different maintenance tasks is taken into account. In the second phase we performed a replicated assessment of the effort prediction models built in the previous phase on a new corrective maintenance project conducted by the subject company on a software system of the same type as the systems of the previous maintenance projects. The data available for the new project were finer grained, according to the indications devised in the first study. This allowed to improve the confidence in our previous empirical analysis by confirming most of the hypotheses made. The new data also provided other useful indications to better understand the maintenance process of the company in a quantitative way.

[1]  D. Rubin,et al.  Statistical Analysis with Missing Data. , 1989 .

[2]  Ingunn Myrtveit,et al.  A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models , 1999, IEEE Trans. Software Eng..

[3]  Allen S. Parrish,et al.  An Empirical Study Using Task Assignment Patterns to Improve the Accuracy of Software Effort Estimation , 2001, IEEE Trans. Software Eng..

[4]  A. S. M. Sajeev,et al.  A Vector-Based Approach to Software Size Measurement and Effort Estimation , 2001, IEEE Trans. Software Eng..

[5]  Lionel C. Briand,et al.  A replicated assessment and comparison of common software cost modeling techniques , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[6]  Victor R. Basili,et al.  A Pattern Recognition Approach for Software Engineering Data Analysis , 1992, IEEE Trans. Software Eng..

[7]  Victor R. Basili,et al.  Understanding and predicting the process of software maintenance releases , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[8]  Ramkumar Ramaswamy,et al.  How to staff business-critical maintenance projects , 2000, IEEE Software.

[9]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[10]  Paul W. Oman,et al.  Using metrics to evaluate software system maintainability , 1994, Computer.

[11]  Danilo Caivano,et al.  Software renewal process comprehension using dynamic effort estimation , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[12]  Magne Jørgensen,et al.  Experience With the Accuracy of Software Maintenance Task Effort Prediction Models , 1995, IEEE Trans. Software Eng..

[13]  Mikael Lindvall Monitoring and Measuring the Change-Prediction Process at Different Granularity Levels , 1998 .

[14]  Rajiv D. Banker,et al.  Software complexity and maintenance costs , 1993, CACM.

[15]  Andrea De Lucia,et al.  Assessing the maintenance processes of a software organization: an empirical analysis of a large industrial project , 2003, J. Syst. Softw..

[16]  R. H. Myers Classical and modern regression with applications , 1986 .

[17]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[18]  Paolo Nesi,et al.  Estimation and Prediction Metrics for Adaptive Maintenance Effort of Object-Oriented Systems , 2001, IEEE Trans. Software Eng..

[19]  Hans van Vliet,et al.  Two case studies in measuring software maintenance effort , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[20]  Taghi M. Khoshgoftaar,et al.  The Detection of Fault-Prone Programs , 1992, IEEE Trans. Software Eng..

[21]  D. Ross Jeffery,et al.  A comparative study of two software development cost modeling techniques using multi-organizational and company-specific data , 2000, Inf. Softw. Technol..

[22]  M. Kendall,et al.  Kendall's advanced theory of statistics , 1995 .

[23]  Khaled El Emam,et al.  Software Cost Estimation with Incomplete Data , 2001, IEEE Trans. Software Eng..

[24]  Stephen G. MacDonell,et al.  A comparison of techniques for developing predictive models of software metrics , 1997, Inf. Softw. Technol..

[25]  William A. Florac,et al.  Measuring the Software Process: Statistical Process Control for Software Process Improvement , 1999 .

[26]  Norman E. Fenton,et al.  A Critique of Software Defect Prediction Models , 1999, IEEE Trans. Software Eng..

[27]  Paolo Nesi,et al.  Managing OO Projects Better , 1998, IEEE Softw..

[28]  Taghi M. Khoshgoftaar,et al.  Modeling the relationship between source code complexity and maintenance difficulty , 1994, Computer.

[29]  Michael J. Prietula,et al.  Software-Effort Estimation: An Exploratory Study of Expert Performance , 1991, Inf. Syst. Res..

[30]  Meir M. Lehman,et al.  Program evolution: processes of software change , 1985 .

[31]  Barbara A. Kitchenham,et al.  Effort estimation using analogy , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[32]  Barry W. Boehm,et al.  Cost models for future software life cycle processes: COCOMO 2.0 , 1995, Ann. Softw. Eng..

[33]  Andrea De Lucia,et al.  A decisional framework for legacy system management , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[34]  Hans van Vliet,et al.  Predicting maintenance effort with function points , 1997, 1997 Proceedings International Conference on Software Maintenance.

[35]  Donald J. Wheeler,et al.  Understanding Statistical Process Control , 1986 .

[36]  Martin J. Shepperd,et al.  Comparing Software Prediction Techniques Using Simulation , 2001, IEEE Trans. Software Eng..

[37]  Chris F. Kemerer,et al.  An Empirical Approach to Studying Software Evolution , 1999, IEEE Trans. Software Eng..

[38]  D. Ross Jeffery,et al.  Cost Estimation : A Review of Models , Process , and Practice , 2010 .

[39]  Lerina Aversano,et al.  Automating the management of software maintenance workflows in a large software enterprise: a case study , 2002, J. Softw. Maintenance Res. Pract..

[40]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[41]  Walter Freiberger,et al.  Statistical Computer Performance Evaluation , 1972 .

[42]  Audris Mockus,et al.  Does Code Decay? Assessing the Evidence from Change Management Data , 2001, IEEE Trans. Software Eng..