Improving dynamic calibration through statistical process control

Dynamic calibration (DC), presented by the authors in previous works has proved to be a flexible approach for massive maintenance software project estimation, able to recalibrate an estimation model in use according to relevant process performance changes pointed out by the project manager. Nevertheless, it results quite subjective in its application and tightly based on manager experience. In this work the authors present an improvement of the approach based on the use of statistical process control (SPC) technique. SPC is a statistically based method able to quickly highlight shift in process performances. It is well known in manufacturing contexts and it has recently emerged in the software engineering community. In this work, authors have integrated SPC in DC as decision support tool for identifying when recalibration of the estimation model must be carried out. This extension makes DC less "person-based", more deterministic and transferable in its use than the previous version. The extended approach has been experimented on industrial data related to a renewal project and the results compared with both, a concurrent approach such as analogy based estimation and its previous version. The results are encouraging and stimulate further investigation.

[1]  Donald J. Wheeler,et al.  Understanding Statistical Process Control , 1986 .

[2]  D. Ross Jeffery,et al.  Using public domain metrics to estimate software development effort , 2001, Proceedings Seventh International Software Metrics Symposium.

[3]  Barbara A. Kitchenham,et al.  Estimates, Uncertainty, and Risk , 1997, IEEE Softw..

[4]  Magne Jørgensen,et al.  Software effort estimation by analogy and "regression toward the mean" , 2003, J. Syst. Softw..

[5]  Donald J. Reifer,et al.  Web Development: Estimating Quick-to-Market Software , 2000, IEEE Softw..

[6]  Barry W. Boehm,et al.  Disaggregating and Calibrating the CASE Tool Variable in COCOMO II , 2002, IEEE Trans. Software Eng..

[7]  Michelle Cartwright,et al.  Predicting with Sparse Data , 2001, IEEE Trans. Software Eng..

[8]  Barry W Boehm,et al.  Software Estimation Perspectives - Guest Editors' Introduction , 2000, IEEE Softw..

[9]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[10]  Amer Baghdadi,et al.  Combining a Performance Estimation Methodology with a Hardware/Software Codesign Flow Supporting Multiprocessor Systems , 2002, IEEE Trans. Software Eng..

[11]  Pankaj Jalote,et al.  Optimum Control Limits for Employing Statistical Process Control in Software Process , 2002, IEEE Trans. Software Eng..

[12]  Mark C. Paulk APPLYING SPC TO THE PERSONAL SOFTWARE PROCESS , 2000 .

[13]  Barry W. Boehm,et al.  Bayesian Analysis of Empirical Software Engineering Cost Models , 1999, IEEE Trans. Software Eng..

[14]  Gavin R. Finnie,et al.  Using Artificial Neural Networks and Function Points to Estimate 4GL Software Development Effort , 1994, Australas. J. Inf. Syst..

[15]  Shari Lawrence Pfleeger,et al.  An empirical study of maintenance and development estimation accuracy , 2002, J. Syst. Softw..

[16]  Marvin V. Zelkowitz,et al.  Experimental Models for Validating Technology , 1998, Computer.

[17]  Isabella Wieczorek,et al.  How valuable is company-specific data compared to multi-company data for software cost estimation? , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[18]  Maria Teresa Baldassarre,et al.  Software renewal projects estimation using dynamic calibration , 2003, International Conference on Software Maintenance, 2003. ICSM 2003. Proceedings..

[19]  Maria Teresa Baldassarre,et al.  Managing Software Process Improvement (SPI) through Statistical Process Control (SPC) , 2004, PROFES.

[20]  L. H. Putnam The Influence Of The Time - Difficulty Factor In Large Scale Software Development , 1977 .

[21]  Ronald Gulezian Reformulating and calibrating COCOMO , 1991, J. Syst. Softw..

[22]  Alessandro Bianchi,et al.  Quality models reuse: experimentation on field , 2002, Proceedings 26th Annual International Computer Software and Applications.

[23]  James H. Cross,et al.  Reverse engineering and design recovery: a taxonomy , 1990, IEEE Software.

[24]  Stephen G. MacDonell,et al.  Combining techniques to optimize effort predictions in software project management , 2003, J. Syst. Softw..

[25]  D. Ross Jeffery,et al.  Calibrating estimation tools for software development , 1990, Softw. Eng. J..

[26]  Barbara A. Kitchenham,et al.  Effort estimation using analogy , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[27]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[28]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[29]  William A. Florac,et al.  Statistical Process Control: Analyzing a Space Shuttle Onboard Software Process , 2000, IEEE Softw..

[30]  Gavin R. Finnie,et al.  Estimating software development effort with connectionist models , 1997, Inf. Softw. Technol..

[31]  T. Saaty Highlights and critical points in the theory and application of the Analytic Hierarchy Process , 1994 .

[32]  F. J. Heemstra,et al.  Software cost estimation , 1992, Inf. Softw. Technol..

[33]  Giuseppe Visaggio,et al.  Analyzing the application of a reverse engineering process to a real situation , 1994, Proceedings 1994 IEEE 3rd Workshop on Program Comprehension- WPC '94.

[34]  Keith Phalp,et al.  An investigation of machine learning based prediction systems , 2000, J. Syst. Softw..

[35]  Giuseppe Visaggio,et al.  Analyzing empirical data from a reverse engineering project , 1995, Proceedings of 2nd Working Conference on Reverse Engineering.

[36]  Douglas Fisher,et al.  Machine Learning Approaches to Estimating Software Development Effort , 1995, IEEE Trans. Software Eng..

[37]  Edward F. Weller Practical Applications of Statistical Process Control , 2000, IEEE Softw..

[38]  Douglas C. Montgomery,et al.  Using statistical control charts for software quality control , 1987 .

[39]  Emilia Mendes,et al.  A Comparative Study of Cost Estimation Models for Web Hypermedia Applications , 2003, Empirical Software Engineering.

[40]  A. R. Crathorne,et al.  Economic Control of Quality of Manufactured Product. , 1933 .

[41]  Barbara Kitchenham,et al.  The MERMAID Approach to software cost estimation , 1990 .

[42]  William A. Florac,et al.  Measuring the Software Process: Statistical Process Control for Software Process Improvement , 1999 .

[43]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[44]  R. F.,et al.  Statistical Method from the Viewpoint of Quality Control , 1940, Nature.

[45]  Barbara A. Kitchenham,et al.  Empirical studies of assumptions that underlie software cost-estimation models , 1992, Inf. Softw. Technol..

[46]  Giuseppe Visaggio,et al.  Journal of Software Maintenance and Evolution: Research and Practice Ageing of a Data-intensive Legacy System: Symptoms and Remedies , 2022 .

[47]  Magne Jørgensen,et al.  A review of studies on expert estimation of software development effort , 2004, J. Syst. Softw..

[48]  Vijay K. Vaishnavi,et al.  Predicting Maintenance Performance Using Object-Oriented Design Complexity Metrics , 2003, IEEE Trans. Software Eng..

[49]  Danilo Caivano,et al.  Software renewal process comprehension using dynamic effort estimation , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.