Revisiting the Conclusion Instability Issue in Software Effort Estimation (S)

Conclusion instability is the absence of observing the same effect under varying experimental conditions. Deep Neural Network (DNN) and ElasticNet software effort estimation (SEE) models were applied to two SEE datasets with the view of resolving the conclusion instability issue and assessing the suitability of ElasticNet as a viable SEE benchmark model. Results were mixed as both model types attain conclusion stability for the Kitchenham dataset whilst conclusion instability existed in the Desharnais dataset. ElasticNet was outperformed by DNN and as such it is not recommended to be used as a SEE benchmark model.

[1]  Pearl Brereton,et al.  Robust Statistical Methods for Empirical Software Engineering , 2017, Empirical Software Engineering.

[2]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[3]  Tim Menzies,et al.  Kernel methods for software effort estimation , 2011, Empirical Software Engineering.

[4]  Jacky W. Keung,et al.  Duplex output software effort estimation model with self-guided interpretation , 2018, Inf. Softw. Technol..

[5]  Tim Menzies,et al.  Special issue on repeatable results in software engineering prediction , 2012, Empirical Software Engineering.

[6]  Karen T. Lum,et al.  Stable rankings for different effort models , 2010, Automated Software Engineering.

[7]  Carolyn Mair,et al.  The consistency of empirical comparisons of regression and analogy-based software project cost prediction , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[8]  Stephen G. MacDonell,et al.  Investigating the Significance of Bellwether Effect to Improve Software Effort Estimation , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[9]  Burak Turhan,et al.  On the dataset shift problem in software engineering prediction models , 2011, Empirical Software Engineering.

[10]  Barbara A. Kitchenham,et al.  A Simulation Study of the Model Evaluation Criterion MMRE , 2003, IEEE Trans. Software Eng..