Practical challenges of requirements prioritization based on risk estimation

Requirements prioritization and risk estimation are known to be difficult. However, so far, risk-based requirements prioritization has not been investigated empirically and quantitatively. In two quantitative experiments, we explored practical challenges and needs of risk estimations in general and of our method MOQARE specifically. In the first experiment, ten students made individual estimations. In the second one, twenty-four students estimated risks in seven moderated groups. The students prioritized the same requirements with different methods (risk estimation and ranking). During the first experiment, we identified factors which influence the quality of the prioritization. In the second experiment, the results of the risk estimation could be improved by discussing risk estimations in a group of experts, gathering risk statistics, and defining requirements, risks and prioritization criteria more tangibly. This first quantitative study on risk-based requirements prioritization helps to understand the practical challenges of this task and thus can serve as a basis for further research on this topic.

[1]  Tim Menzies,et al.  Experiences using Visualization Techniques to Present Requirements, Risks to Them, and Options for Risk Mitigation , 2006, 2006 First International Workshop on Requirements Engineering Visualization (REV'06 - RE'06 Workshop).

[2]  R. Hyman.  Quasi-Experimentation: Design and Analysis Issues for Field Settings (Book) , 1982 .

[3]  Patrik Berander,et al.  Prioritization of Stakeholder Needs in Software Engineering : Understanding and Evaluation , 2004 .

[4]  Rahul Telang,et al.  An ounce of prevention vs. a pound of cure: How can we measure the value of IT security solutions? , 2004 .

[5]  C W Simcoe,et al.  An ounce of prevention. , 1978, Journal - American Intra-Ocular Implant Society.

[6]  Antonis C. Stylianou,et al.  A total quality management-based systems development process , 1997, DATB.

[7]  Claes Wohlin,et al.  An evaluation of methods for prioritizing software requirements , 1998, Inf. Softw. Technol..

[8]  Barry Boehm,et al.  Bayesian analysis of software cost and quality models , 1999 .

[9]  Jane Cleland-Huang,et al.  Software by Numbers - Low-Risk, High-Return Development , 2003 .

[10]  Alan M. Davis,et al.  The Art of Requirements Triage , 2003, Computer.

[11]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[12]  Allen H. Dutoit,et al.  A Rationale-based Analysis Tool , 2004, IASSE.

[13]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[14]  Ken Frazer,et al.  Review of "Managing software requirements, a use case approach by Dean Leffingwell and Don Widrig." Addison-Wesley 2003 , 2004, SOEN.

[15]  Andreas L. Opdahl,et al.  Eliciting security requirements with misuse cases , 2000, Proceedings 37th International Conference on Technology of Object-Oriented Languages and Systems. TOOLS-Pacific 2000.

[16]  Sjaak Brinkkemper,et al.  Flexible Release Composition using Integer Linear Programming , 2004 .

[17]  Tim Menzies,et al.  Optimizing requirements decisions with keys , 2008, PROMISE '08.

[18]  Barbara Paech,et al.  Exploring the Interoperability of Web Services using MOQARE , 2010 .

[19]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[20]  Tim Menzies,et al.  Improved Software Engineering Decision Support Through Automatic Argument Reduction Tools , 2003, SEKE.

[21]  Barbara Paech,et al.  Eliciting and Maintaining Knowledge for Requirements Evolution , 2003 .

[22]  Barry W. Boehm,et al.  Bayesian Analysis of Empirical Software Engineering Cost Models , 1999, IEEE Trans. Software Eng..

[23]  Martin S. Feather,et al.  Combining the best attributes of qualitative and quantitative risk management tool support , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[24]  Martin S. Feather,et al.  Quantitative risk-based requirements reasoning , 2003, Requirements Engineering.

[25]  Jeffrey C. Carver,et al.  Observational studies to accelerate process experience in classroom studies: an evaluation , 2003, 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings..

[26]  E. Dubois,et al.  Towards a Risk-Based Security Requirements Engineering Framework , 2005 .

[27]  Martin Höst,et al.  An Industrial Case Study on Distributed Prioritisation in Market-Driven Requirements Engineering for Packaged Software , 2001, Requirements Engineering.

[28]  Barbara Paech,et al.  Quality Misuse , 2005 .

[29]  Claes Wohlin,et al.  Engineering and Managing Software Requirements , 2005 .

[30]  Claes Wohlin,et al.  Pair-wise comparisons versus planning game partitioning—experiments on requirements prioritisation techniques , 2007, Empirical Software Engineering.

[31]  Walter F. Tichy,et al.  Hints for Reviewing Empirical Work in Software Engineering , 2000, Empirical Software Engineering.

[32]  Scott W. Ambler,et al.  Agile modeling: effective practices for extreme programming and the unified process , 2002 .

[33]  Barbara Paech,et al.  Rationale-Based Use Case Specification , 2002, Requirements Engineering.

[34]  Colette Rolland,et al.  Payoff Analysis of Business Systems in Goal-Oriented Requirements Engineering , 2004 .

[35]  Karl E. Wiegers.  First Things First: Prioritizing Requirements , 1999 .

[36]  Nancy R. Mead,et al.  SQUARE Project: Cost/Benefit Analysis Framework for Information Security Improvement Projects in Small Companies , 2004 .

[37]  H. Raiffa,et al.  Negotiation Analysis: The Science and Art of Collaborative Decision Making , 2003 .

[38]  Martin P. Loeb,et al.  CSI/FBI Computer Crime and Security Survey , 2004 .

[39]  Patrik Berander,et al.  Prioritization of Stakeholder Needs in Software Engineering , 2004 .

[40]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[41]  Claes Wohlin,et al.  Using Students as Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment , 2000, Empirical Software Engineering.

[42]  William Remus,et al.  Using students as subjects in experiments on decision support systems , 1989, [1989] Proceedings of the Twenty-Second Annual Hawaii International Conference on System Sciences. Volume III: Decision Support and Knowledge Based Systems Track.

[43]  Jyrki Kontio,et al.  The Riskit Method for Software Risk Management, version 1.00 , 1997 .

[44]  Sjaak Brinkkemper,et al.  Software product release planning through optimization and what-if analysis , 2008, Inf. Softw. Technol..

[45]  Steve McConnell.  From the Editor - An Ounce of Prevention , 2001, IEEE Softw..

[46]  H. Schneider.  Failure mode and effect analysis : FMEA from theory to execution , 1996 .

[47]  Andrea Herrmann,et al.  Requirements Prioritization Based on Benefit and Cost Prediction: A Method Classification Framework , 2008, 2008 34th Euromicro Conference Software Engineering and Advanced Applications.

[48]  Günther Ruhe,et al.  Decision Support in Requirements Engineering , 2005 .

[49]  Sunita Chulani,et al.  Bayesian analysis of software cost and quality models , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[50]  R. Power.  CSI/FBI computer crime and security survey , 2001 .

[51]  Barbara Paech,et al.  MOQARE: misuse-oriented quality requirements engineering , 2008, Requirements Engineering.

[52]  Joachim Karlsson,et al.  Software requirements prioritizing , 1996, Proceedings of the Second International Conference on Requirements Engineering.

[53]  Per Runeson,et al.  Using Students as Experiment Subjects – An Analysis on Graduate and Freshmen Student Data , 2003 .

[54]  Dietmar Pfahl,et al.  Trade-off Analysis for Requirements Selection , 2003, Int. J. Softw. Eng. Knowl. Eng..

[55]  Dean Leffingwell,et al.  Managing software requirements: a unified approach , 1999 .

[56]  Claes Wohlin,et al.  Requirements prioritisation: an experiment on exhaustive pair-wise comparisons versus planning game partitioning , 2004, ICSE 2004.

[57]  Patrik Berander,et al.  Hierarchical Cumulative Voting (hcv) - Prioritization of Requirements in Hierarchies , 2006, Int. J. Softw. Eng. Knowl. Eng..

[58]  Meliha Handzic,et al.  Managing Software Engineering Knowledge , 2010, Springer Berlin Heidelberg.

[59]  Jung-Won Park.  Supporting Distributed Collaborative Prioritization for WinWin Requirements Capture and Negotiations , 1999 .

[60]  Joachim Karlsson,et al.  Prioritizing Software Requirements In An Industrial Setting , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[61]  Rahul Telang,et al.  Measuring the risk-based value of IT security solutions , 2004, IT Professional.

[62]  K. Beck,et al.  Extreme Programming Explained , 2002 .

[63]  Martin S. Feather,et al.  Scalable mechanisms for requirements interaction management , 2000, Proceedings Fourth International Conference on Requirements Engineering. ICRE 2000. (Cat. No.98TB100219).

[64]  J. Knottnerus,et al.  Real world research. , 2010, Journal of clinical epidemiology.

[65]  Andreas L. Opdahl,et al.  Templates for Misuse Case Description , 2001 .