Studying the impact of uncertainty in operational release planning - An integrated method and its initial evaluation

Context: Uncertainty is an unavoidable issue in software engineering and an important area of investigation. This paper studies the impact of uncertainty on total duration (i.e., make-span) for implementing all features in operational release planning. Objective: The uncertainty factors under investigation are: (1) the number of new features arriving during release construction, (2) the estimated effort needed to implement features, (3) the availability of developers, and (4) the productivity of developers. Method: An integrated method is presented combining Monte-Carlo simulation (to model uncertainty in the operational release planning (ORP) process) with process simulation (to model the ORP process steps and their dependencies as well as an associated optimization heuristic representing an organization-specific staffing policy for make-span minimization). The method allows for evaluating the impact of uncertainty on make-span. The impact of uncertainty factors both in isolation and in combination are studied in three different pessimism levels through comparison with a baseline plan. Initial evaluation of the method is done by an explorative case study at Chartwell Technology Inc. to demonstrate its applicability and its usefulness. Results: The impact of uncertainty on release make-span increases - both in terms of magnitude and variance - with an increase of pessimism level as well as with an increase of the number of uncertainty factors. Among the four uncertainty factors, we found that the strongest impact stems from the number of new features arriving during release construction. We have also demonstrated that for any combination of uncertainty factors their combined (i.e., simultaneous) impact is bigger than the addition of their individual impacts. Conclusion: The added value of the presented method is that managers are able to study the impact of uncertainty on existing (i.e., baseline) operational release plans pro-actively.

[1]  Barbara A. Kitchenham,et al.  Experiences of using an evaluation framework , 2005, Inf. Softw. Technol..

[2]  Arthur Bloch,et al.  マーフィーの法則 : 現代アメリカの知性 = Murphy's law , 1993 .

[3]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[4]  Dietmar Pfahl,et al.  A System Dynamics Simulation Model for Analyzing the Stability of Software Release Plans: Research Sections , 2007 .

[5]  Henry F. Ledgard,et al.  Measuring productivity in the software industry , 2003, CACM.

[6]  Dietmar Pfahl,et al.  Simulating Worst Case Scenarios and Analyzing Their Combined Effect in Operational Release Planning , 2008, ICSP.

[7]  Rainer Kolisch,et al.  Experimental investigation of heuristics for resource-constrained project scheduling: An update , 2006, Eur. J. Oper. Res..

[8]  Barry W. Boehm,et al.  Value-based software engineering: reinventing , 2003, SOEN.

[9]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[10]  Dietmar Pfahl,et al.  Software Process Simulation Modeling: Facts, Trends and Directions , 2008, 2008 15th Asia-Pacific Software Engineering Conference.

[11]  Jan Węglarz,et al.  Project scheduling : recent models, algorithms, and applications , 1999 .

[12]  Giuliano Antoniol,et al.  A robust search-based approach to project management in the presence of abandonment, rework, error and uncertainty , 2004 .

[13]  R. Kolisch,et al.  Heuristic algorithms for the resource-constrained project scheduling problem: classification and computational analysis , 1999 .

[14]  Giuliano Antoniol,et al.  Search-based techniques applied to optimization of project planning for a massive maintenance project , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[15]  Jim Duggan,et al.  A task allocation optimizer for software construction , 2004, IEEE Software.

[16]  M.Ricafranca Romulo.,et al.  A STUDY OF UNCERTAINTY , 1970 .

[17]  Bo Yang,et al.  A Study of Uncertainty in Software Cost and Its Impact on Optimal Software Release Time , 2008, IEEE Transactions on Software Engineering.

[18]  Dietmar Pfahl,et al.  A method for re-planning of software releases using discrete-event simulation , 2008, Softw. Process. Improv. Pract..

[19]  David Johnson,et al.  The triangular distribution as a proxy for the beta distribution in risk analysis , 1997 .

[20]  Natalia Juristo Juzgado,et al.  Emphasizing human capabilities in software development , 2006, IEEE Software.

[21]  Dietmar Pfahl,et al.  A System Dynamics Simulation Model for Analyzing the Stability of Software Release Plans , 2007, Softw. Process. Improv. Pract..

[22]  Dietmar Pfahl,et al.  ProSim/RA - Software Process Simulation in Support of Risk Assessment , 2006, Value-Based Software Engineering.

[23]  Dietmar Pfahl,et al.  A method for re-planning of software releases using discrete-event simulation , 2008 .

[24]  June M. Verner,et al.  State of the practice: An exploratory analysis of schedule estimation and software project success prediction , 2007, Inf. Softw. Technol..

[25]  George E. Stark,et al.  An examination of the effects of requirements changes on software maintenance releases , 1999, J. Softw. Maintenance Res. Pract..

[26]  Frank Padberg.  Computing optimal scheduling policies for software projects , 2004, 11th Asia-Pacific Software Engineering Conference.

[27]  Günther Ruhe,et al.  Optimized Resource Allocation for Software Release Planning , 2009, IEEE Transactions on Software Engineering.

[28]  Frank Padberg.  On the potential of process simulation in software project schedule optimization , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[29]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[30]  Magne Jørgensen,et al.  The role of outcome feedback in improving the uncertainty assessment of software development effort estimates , 2008, TSEM.

[31]  Efraim Turban,et al.  Decision support systems and intelligent systems , 1997 .

[32]  Tao Zhang,et al.  Genetic Algorithms for Project Management , 2001, Ann. Softw. Eng..

[33]  Frank Padberg.  Using process simulation to compare scheduling strategies for software projects , 2002, Ninth Asia-Pacific Software Engineering Conference, 2002..

[34]  Dietmar Pfahl,et al.  SIMULATION-BASED UNCERTAINTY ANALYSIS FOR PLANNING PARAMETERS IN OPERATIONAL PRODUCT MANAGEMENT , 2007 .

[35]  George E. Stark,et al.  An examination of the effects of requirements changes on software maintenance releases , 1999 .

[36]  Raymond J. Madachy,et al.  Software process simulation modeling: Why? What? How? , 1999, J. Syst. Softw..

[37]  Barbara A. Kitchenham,et al.  A framework for evaluating a software bidding model , 2005, Inf. Softw. Technol..

[38]  Andrew Smith,et al.  Optimized staffing for product releases and its application at Chartwell Technology , 2008, J. Softw. Maintenance Res. Pract..