Search based data sensitivity analysis applied to requirement engineering

Software engineering is plagued by problems associated with unreliable cost estimates. This paper introduces an approach to sensitivity analysis for requirements engineering. It uses Search-Based Software Engineering to aid the decision maker to explore sensitivity of the cost estimates of requirements for the Next Release Problem (NRP). The paper presents both single- and multi-objective formulation of NRP with empirical sensitivity analysis on synthetic and real-world data. The results show strong correlation between the level of inaccuracy and the impact on the selection of requirements, as well as between the cost of requirements and the impact, which is as intuitively expected. However, there also exist a few sensitive exceptions to these trends; the paper uses a heat-map style visualisation to reveal these exceptions which require careful consideration. The paper also shows that such unusually sensitivity patterns occur in real-world data and how the proposed approach clearly identifies them.

[1]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[2]  Kalyanmoy Deb,et al.  Searching for Robust Pareto-Optimal Solutions in Multi-objective Optimization , 2005, EMO.

[3]  Barry W. Boehm,et al.  Software development cost estimation approaches — A survey , 2000, Ann. Softw. Eng..

[4]  Mark Harman,et al.  Search Based Approaches to Component Selection and Prioritization for the Next Release Problem , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[5]  Charalampos Baniotopoulos,et al.  A contribution to the sensitivity analysis of the sea-bed-structure interaction problem for underwater pipelines , 1991 .

[6]  H Christopher Frey,et al.  OF SENSITIVITY ANALYSIS , 2001 .

[7]  R. Levine,et al.  A Sensitivity Analysis of Cross-Country Growth Regressions , 1991 .

[8]  Phillip G. Armour,et al.  Ten unmyths of project estimation , 2002, CACM.

[9]  Andrzej Osyczka,et al.  7 – Multicriteria optimization for engineering design , 1985 .

[10]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[11]  Aravind Seshadri,et al.  A FAST ELITIST MULTIOBJECTIVE GENETIC ALGORITHM: NSGA-II , 2000 .

[12]  Victor J. Rayward-Smith,et al.  The next release problem , 2001, Inf. Softw. Technol..

[13]  Søren L. Buhl,et al.  How (In)accurate Are Demand Forecasts in Public Works Projects?: The Case of Transportation , 2005, 1303.6654.

[14]  Yang Cao,et al.  Sensitivity analysis of discrete stochastic systems. , 2005, Biophysical journal.

[15]  Yuanyuan Zhang,et al.  The multi-objective next release problem , 2007, GECCO '07.

[16]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithm test suites , 1999, SAC '99.

[17]  Søren L. Buhl,et al.  Underestimating Costs in Public Works Projects: Error or Lie? , 2002, 1303.6604.

[18]  Sjaak Brinkkemper,et al.  Software product release planning through optimization and what-if analysis , 2008, Inf. Softw. Technol..

[19]  Andrzej Ameljańczyk,et al.  Multicriteria Optimization in Engineering Design , 1994 .

[20]  Joseph A. C. Delaney Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.