Robust next release problem: handling uncertainty during optimization

Uncertainty is inevitable in real world requirement engineering. It has a significant impact on the feasibility of proposed solutions and thus brings risks to the software release plan. This paper proposes a multi-objective optimization technique, augmented with Monte-Carlo Simulation, that optimizes requirement choices for the three objectives of cost, revenue, and uncertainty. The paper reports the results of an empirical study over four data sets derived from a single real world data set. The results show that the robust optimal solutions obtained by our approach are conservative compared to their corresponding optimal solutions produced by traditional Multi-Objective Next Release Problem. We obtain a robustness improvement of at least 18% at a small cost (a maximum 0.0285 shift in the 2D Pareto-front in the unit space). Surprisingly we found that, though a requirement's cost is correlated with inclusion on the Pareto-front, a requirement's expected revenue is not.

[1]  Yuanyuan Zhang,et al.  The multi-objective next release problem , 2007, GECCO '07.

[2]  Mark Harman,et al.  Search Based Approaches to Component Selection and Prioritization for the Next Release Problem , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[3]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[4]  Dietmar Pfahl,et al.  Decision Support for Product Release Planning Based on Robustness Analysis , 2010, 2010 18th IEEE International Requirements Engineering Conference.

[5]  Rémi Bardenet,et al.  Monte Carlo Methods , 2013, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[6]  Des Greer,et al.  Software release planning: an evolutionary and iterative approach , 2004, Inf. Softw. Technol..

[7]  Earl T. Barr,et al.  Uncertainty, risk, and information value in software requirements and architecture , 2014, ICSE.

[8]  Günther Ruhe,et al.  Search Based Software Engineering , 2013, Lecture Notes in Computer Science.

[9]  Victor J. Rayward-Smith,et al.  The next release problem , 2001, Inf. Softw. Technol..

[10]  Z Zi,et al.  Sensitivity analysis approaches applied to systems biology models. , 2011, IET systems biology.

[11]  Jerffeson Souza,et al.  A scenario-based robust model for the next release problem , 2013, GECCO '13.

[12]  Stefano Tarantola,et al.  Sensitivity Analysis as an Ingredient of Modeling , 2000 .

[13]  Mark Harman,et al.  Exact scalable sensitivity analysis for the next release problem , 2014, ACM Trans. Softw. Eng. Methodol..

[14]  Dietmar Pfahl,et al.  Studying the impact of uncertainty in operational release planning - An integrated method and its initial evaluation , 2010, Inf. Softw. Technol..

[15]  B. Flyvbjerg,et al.  Why Your IT Project May Be Riskier than You Think , 2011 .

[16]  Bernhard Sendhoff,et al.  Robust Optimization - A Comprehensive Survey , 2007 .

[17]  Shin Yoo,et al.  Search based data sensitivity analysis applied to requirement engineering , 2009, GECCO.

[18]  Melvyn Sim,et al.  The Price of Robustness , 2004, Oper. Res..

[19]  Enrique Alba,et al.  Today/future importance analysis , 2010, GECCO '10.

[20]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[21]  G. Nemhauser,et al.  Discrete Dynamic Programming and Capital Allocation , 1969 .

[22]  Yuanyuan Zhang,et al.  “Fairness Analysis” in Requirements Assignments , 2008, 2008 16th IEEE International Requirements Engineering Conference.

[23]  Debra J. Richardson,et al.  The Uncertainty Principle in Software Engineering Submitted to ICSE'97, 19th International Conference on Software Engineering , 1998 .

[24]  Shapour Azarm,et al.  A multi-objective genetic algorithm for robust design optimization , 2005, GECCO '05.

[25]  Gordon B. Hazen,et al.  Sensitivity Analysis and the Expected Value of Perfect Information , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.