Realism in assessment of effort estimation uncertainty: it matters how you ask

Traditionally, software professionals are requested to provide minimum-maximum intervals to indicate the uncertainty of their effort estimates. We claim that the traditional request is not optimal and leads to overoptimistic views about the level of estimation uncertainty. Instead, we propose that it is better to frame the request for uncertainty assessment: "How likely is it that the actual effort will be more than/less than X?" Our claim is based on the results of a previously reported-experiment and field studies in two companies. The two software companies were instructed to apply the traditional and our alternative framing on random samples of their projects. In total, we collected information about 47 projects applying the traditional-framing and 23 projects applying the alternative framing.

[1]  JørgensenMagne,et al.  Expert Estimation of Web-Development Projects , 2005 .

[2]  Terry Connolly,et al.  Decomposed versus holistic estimates of effort required for software writing tasks , 1997 .

[3]  Magne Jørgensen Top-down and bottom-up expert estimation of software development effort , 2004, Inf. Softw. Technol..

[4]  A. Ehrenberg,et al.  The Design of Replicated Studies , 1993 .

[5]  E. W. Davis,et al.  Project Management With Cpm, Pert and Precedence Diagramming , 1983 .

[6]  Dag I. K. Sjøberg,et al.  The impact of customer expectation on software development effort estimates , 2004 .

[7]  A. Tversky,et al.  Variants of uncertainty , 1982, Cognition.

[8]  Magne Jørgensen,et al.  Expert Estimation of Web-Development Projects: Are Software Professionals in Technical Roles More Optimistic Than Those in Non-Technical Roles? , 2004, Empirical Software Engineering.

[9]  Tariq Haroon Ahmad,et al.  Project management with CPM , 1976 .

[10]  Dean P. Foster,et al.  Precision and Accuracy of Judgmental Estimation , 1997 .

[11]  Magne Jørgensen,et al.  Better sure than safe? Over-confidence in judgement based software development effort prediction intervals , 2004, J. Syst. Softw..

[12]  Thomas Gilovich,et al.  Just Going Along: Nonconscious Priming and Conformity to Social Pressure , 1999 .

[13]  Building a software cost estimation model based on categorical data , 2001, Proceedings Seventh International Software Metrics Symposium.

[14]  M. Jorgensen,et al.  Uncertainty Intervals versus Interval Uncertainty: An Alternative Method for Eliciting Effort Prediction Intervals in Software Development Projects (ProMAC2002予稿集) -- (Risk Management(1)) , 2002 .

[15]  Ioannis Stamelos,et al.  A Simulation Tool for Efficient Analogy Based Cost Estimation , 2000, Empirical Software Engineering.

[16]  Magne Jørgensen,et al.  Regression Models of Software Development Effort Estimation Accuracy and Bias , 2004, Empirical Software Engineering.

[17]  David N. Card,et al.  Managers Handbook for Software Development , 1984 .

[18]  Magne Jørgensen,et al.  An effort prediction interval approach based on the empirical distribution of previous estimation accuracy , 2003, Inf. Softw. Technol..

[19]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.