Are Size Measures Better Than Expert Judgment? An Industrial Case Study on Requirements Volatility

Expert judgment is a common estimation approach in industry. However, there is very little research on the accuracy of expert judgment outside the area of effort estimation. In this paper, we present an industrial case study investigating subjective and objective measures of requirementss volatility. Data was collected in retrospect for all use cases of a medium-size software project. In addition, we determined subjective volatility by interviewing developers and managers of the project. Our data analysis show that structural measures perform better than expert judgment in estimating the total number of changes to use case based requirements. These results confirm results from a previous case study by the authors and suggest that project managers should not rely on expert judgment alone for decision making.

[1]  Didar Zowghi,et al.  A study of the impact of requirements volatility on software project performance , 2002, Ninth Asia-Pacific Software Engineering Conference, 2002..

[2]  Magne Jørgensen.  Top-down and bottom-up expert estimation of software development effort , 2004, Inf. Softw. Technol..

[3]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[4]  Geri Schneider,et al.  Applying Use Cases: A Practical Guide , 1998 .

[5]  Joseph Raynus.  Software Process Improvement With CMM , 1998 .

[6]  Barbara Ann Kitchenham,et al.  Evaluating software engineering methods and tools: part 9: quantitative case study methodology , 1998, SOEN.

[7]  Didar Zowghi,et al.  Analysis of requirements volatility during software development life cycle , 2004, 2004 Australian Software Engineering Conference. Proceedings..

[8]  John H. Baumert,et al.  Software Measures and the Capability Maturity Model , 1992 .

[9]  Susan Lilly,et al.  Use case pitfalls: top 10 problems from real projects using use cases , 1999, Proceedings of Technology of Object-Oriented Languages and Systems - TOOLS 30 (Cat. No.PR00278).

[10]  Jürgen Börstler,et al.  Construction and Validation of Prediction Models for Number of Changes to Requirements , 2007 .

[11]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[12]  Vincenzo Gervasi,et al.  Process Metrics for Requirements Analysis , 2000, EWSPT.

[13]  Kevin J. Madders,et al.  EUROPEAN SPACE AGENCY , 1983 .

[14]  Robert T. Hughes,et al.  Expert judgement as an estimating method , 1996, Inf. Softw. Technol..

[15]  Jürgen Börstler,et al.  An industrial case study on requirements volatility measures , 2005, 12th Asia-Pacific Software Engineering Conference (APSEC'05).

[16]  Gary A. Ham.  Four Roads to Use Case Discovery There Is a Use ( and a Case ) for Each One , .

[17]  Lionel C. Briand,et al.  Empirical Studies of Quality Models in Object-Oriented Systems , 2002, Adv. Comput..

[18]  Ralph Young,et al.  The requirements engineering handbook , 2003 .

[19]  Annabella Loconsole,et al.  Definition and validation of requirements management measures , 2007 .

[20]  Lars Lundberg,et al.  The accuracy of fault prediction in modified code - statistical model vs. expert estimation , 2006, 13th Annual IEEE International Symposium and Workshop on Engineering of Computer-Based Systems (ECBS'06).

[21]  Mary Beth Chrissis,et al.  CMMI: Guidelines for Process Integration and Product Improvement , 2003 .

[22]  Ivar Jacobson,et al.  Use cases – Yesterday, today, and tomorrow , 2004, Software & Systems Modeling.

[23]  George E. Stark,et al.  An examination of the effects of requirements changes on software maintenance releases , 1999, J. Softw. Maintenance Res. Pract..

[24]  Lionel C. Briand,et al.  Exploring the relationships between design measures and software quality in object-oriented systems , 2000, J. Syst. Softw..

[25]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[26]  Linda H. Rosenberg,et al.  A Software Quality Model and Metrics for Identifying Project Risks and Assessing Software Quality , 1996 .

[27]  Claes Wohlin,et al.  Assessing Project Success Using Subjective Evaluation Factors , 2004, Software Quality Journal.