Requirements Prioritisation and Retrospective Analysis for Release Planning Process Improvement

The quality of a product can be defined by its ability to satisfy the needs and expectations of its customers. Achieving quality is especially difficult in market-driven situations since the product is released on an open market with numerous potential customers and users with various wishes. The quality of the software product is to a large extent determined by the quality of the requirements engineering (RE) and release planning decisions regarding which requirements that are selected for a product. The goal of this thesis is to enhance software product quality and increase the competitive edge of software organisations by improving release planning decision-making. The thesis is based on empirical research, including both qualitative and quantitative research approaches. The research contains a qualitative survey of RE challenges in market-driven organisations based on interviews with practitioners. The survey provided increased understanding of RE challenges in the software industry and gave input to the continued research. Among the challenging issues, one was selected for further investigation due to its high relevance to the practitioners: requirements prioritisation and release planning decision-making. Requirements prioritisation techniques were evaluated through experiments, suggesting that ordinal scale techniques based on grouping and ranking may be valuable to practitioners. Finally, a retrospective method called PARSEQ (Post-release Analysis of Requirements SElection Quality) is introduced and tested in three case studies. The method aims at evaluating prior releases and finding improvement proposals for release planning decision-making in future release projects. The method was found valuable by all participants and relevant improvement proposals were discovered in all cases. (Less)

[1]  Joachim Karlsson,et al.  A systematic approach for prioritizing software requirements , 1998 .

[2]  Joachim Karlsson,et al.  A Cost-Value Approach for Prioritizing Requirements , 1997, IEEE Softw..

[3]  Alberto L. Sangiovanni-Vincentelli,et al.  Platform-Based Design and Software Design Methodology for Embedded Systems , 2001, IEEE Des. Test Comput..

[4]  Pär Carlshamre,et al.  A usability perspective on requirements engineering : from methodology to product development , 2001 .

[5]  Dietmar Pfahl,et al.  Quantitative WinWin: a new method for decision support in requirements negotiation , 2002, SEKE '02.

[6]  Björn Regnell,et al.  An industrial survey of requirements interdependencies in software product release planning , 2001, Proceedings Fifth IEEE International Symposium on Requirements Engineering.

[7]  Bernhard Deifel.  A process model for requirements engineering of CCOTS , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[8]  M. Patton.  Qualitative research and evaluation methods , 1980 .

[9]  Pete Sawyer,et al.  Packaged Software: Challenges for RE , 2000 .

[10]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[11]  Des Greer,et al.  Software release planning: an evolutionary and iterative approach , 2004, Inf. Softw. Technol..

[12]  Erik Kamsties,et al.  Requirements Engineering in Small and Medium Enterprises: State-of-the-Practice, Problems, Solutions, and Technology Transfer , 1998 .

[13]  Björn Regnell,et al.  A market-driven requirements engineering process: Results from an industrial process improvement programme , 1998, Requirements Engineering.

[14]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[15]  Joachim Karlsson,et al.  Improved practical support for large-scale requirements prioritising , 1997, Requirements Engineering.

[16]  Karl E. Wiegers,et al.  Software Requirements , 1999 .

[17]  Marjo Kauppinen,et al.  Requirements Prioritization Challenges in Practice , 2004, PROFES.

[18]  Mark C. Paulk,et al.  The Capability Maturity Model: Guidelines for Improving the Software Process , 1994 .

[19]  A. Strauss,et al.  The Discovery of Grounded Theory , 1967 .

[20]  Norman L. Kerth,et al.  Project Retrospectives: A Handbook for Team Reviews , 2001 .

[21]  Robert C. Martin,et al.  Extreme programming in practice , 2001, OOPSLA '00.

[22]  Michael D. Myers,et al.  Qualitative Research in Information Systems , 1997, MIS Q..

[23]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[24]  A. E. Hoerl,et al.  An incomplete design in the analytic hierarchy process , 1992 .

[25]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[26]  J. M. Singer,et al.  IEEE Recommended Practice for Software Requirements SpeciÞcations , 1993 .

[27]  Jeffrey C. Carver,et al.  Issues in using students in empirical studies in software engineering education , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[28]  Sjaak Brinkkemper,et al.  A method to draw lessons from project postmortem databases , 2006, Softw. Process. Improv. Pract..

[29]  J. O'Neill,et al.  Project management. , 2001, Health management technology.

[30]  DeMarcoTom,et al.  A Defined Process For Project Postmortem Review , 1996 .

[31]  Tom DeMarco,et al.  A defined process for project post mortem review , 1996, IEEE Software.

[32]  S. Lauesen.  Software Requirements Styles and Techniques , 2001 .

[33]  Robert G. Cooper,et al.  Winning at new products : accelerating the process from idea to launch , 2001 .

[34]  Patrik Berander,et al.  Using students as subjects in requirements prioritization , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[35]  Austen Rainer,et al.  Requirements problems in twelve software companies: an empirical analysis , 2002, IEE Proc. Softw..

[36]  Khaled El Emam,et al.  A field study of requirements engineering practices in information systems development , 1995, Proceedings of 1995 IEEE International Symposium on Requirements Engineering (RE'95).

[37]  Alan M. Davis,et al.  The Art of Requirements Triage , 2003, Computer.

[38]  Martin Höst,et al.  An analytical model for requirements selection quality evaluation in product software development , 2003, Proceedings. 11th IEEE International Requirements Engineering Conference, 2003..

[39]  P. Harker.  Incomplete pairwise comparisons in the analytic hierarchy process , 1987 .

[40]  Martin Höst,et al.  An Industrial Case Study on Distributed Prioritisation in Market-Driven Requirements Engineering for Packaged Software , 2001, Requirements Engineering.

[41]  Thomas L. Saaty,et al.  Multicriteria Decision Making: The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation , 1990 .

[42]  Prodromos D. Chatzoglou,et al.  Factors affecting completion of the requirements capture stage of projects with different characteristics , 1997, Inf. Softw. Technol..

[43]  Robert J. Novorita,et al.  BENEFITS OF STRUCTURED REQUIREMENTS METHODS FOR MARKET‐BASED ENTERPRISES , 1996 .

[44]  Kendall Scott,et al.  UML distilled - a brief guide to the Standard Object Modeling Language (2. ed.) , 2000, notThenot Addison-Wesley object technology series.

[45]  Carina Andersson,et al.  Exploring the Software Verification and Validation Process with Focus on Efficient Fault Detection , 2003 .

[46]  Kevin C. Desouza,et al.  Experiences with conducting project postmortems: reports versus stories , 2005, Softw. Process. Improv. Pract..

[47]  Dean Leffingwell,et al.  Managing software requirements: a unified approach , 1999 .

[48]  Erran Carmel,et al.  A process model for packaged software development , 1995 .

[49]  Standard Glossary of Software Engineering Terminology , 1990 .

[50]  J. Herbsleb,et al.  A systematic survey of CMM experience and results , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[51]  H. D. Rombach,et al.  THE EXPERIENCE FACTORY , 1999 .

[52]  Per Runeson,et al.  Using Students as Experiment Subjects – An Analysis on Graduate and Freshmen Student Data , 2003 .

[53]  Pär Carlshamre,et al.  Release Planning in Market-Driven Software Product Development: Provoking an Understanding , 2002, Requirements Engineering.

[54]  Mark C. Paulk,et al.  Capability Maturity Model for Software , 2001 .

[55]  Jens K Andersson,et al.  Overload control and performance evaluation in a Parlay/OSA environment , 2004 .

[56]  Heikki Kälviäinen,et al.  A State-of-the-Practice Survey on Requirements Engineering in Small- and Medium-Sized Enterprises , 2000 .

[57]  K. Beck,et al.  Extreme Programming Explained , 2002 .

[58]  Claes Wohlin,et al.  Using Students as Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment , 2000, Empirical Software Engineering.

[59]  M. Lindvall,et al.  Knowledge management in software engineering , 2002, IEEE Software.

[60]  Khaled El Emam,et al.  Benchmarking Kappa: Interrater Agreement in Software Process Assessments , 1999, Empirical Software Engineering.

[61]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[62]  Claes Wohlin,et al.  What is important when deciding to include a software requirement in a project or release? , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[63]  Jawed I. A. Siddiqi,et al.  Requirements Engineering: The Emerging Wisdom , 1996, IEEE Softw..

[64]  M. Hoepfl.  Choosing Qualitative Research: A Primer for Technology Education Researchers , 1997 .

[65]  Ivar Jacobson,et al.  The Unified Software Development Process , 1999 .

[66]  Soren Lauesen,et al.  Software Requirements: Styles & Techniques , 2002 .

[67]  Marjo Kauppinen,et al.  Introducing requirements engineering: how to make a cultural change happen in practice , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[68]  Søren Lauesen,et al.  Preventing Requirement Defects: An Experiment in Process Improvement , 2001, Requirements Engineering.

[69]  C. Brodsky.  The Discovery of Grounded Theory: Strategies for Qualitative Research , 1968 .

[70]  Torgeir Dingsøyr,et al.  Postmortem: Never Leave a Project without It , 2002, IEEE Softw..

[71]  Karl T. Ulrich,et al.  Product Design and Development , 1995 .

[72]  Qi Zhang,et al.  A method of evaluation for scaling in the analytic hierarchy process , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).

[73]  Torgeir Dingsøyr,et al.  Postmortem reviews: purpose and approaches in software engineering , 2005, Inf. Softw. Technol..

[74]  Hans Toetenel,et al.  Embedded Software Engineering: The State of the Practice , 2003, IEEE Softw..

[75]  Stewart A. Higgins,et al.  Managing Requirements for Medical IT Products , 2003, IEEE Softw..

[76]  Mark C. Paulk,et al.  Capability Maturity Model for Software, Version 1.1 , 1993 .

[77]  Daniela E. Damian,et al.  RE challenges in multi-site software development organisations , 2003, Requirements Engineering.

[78]  K L Easton,et al.  Avoiding Common Pitfalls in Qualitative Data Collection and Transcription , 2000, Qualitative health research.

[79]  Charles Richter,et al.  A review of the state of the practice in requirements modeling , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.

[80]  Franz Lehner,et al.  Requirements Engineering as a Success Factor in Software Projects , 2001, IEEE Softw..

[81]  Julio Cesar Sampaio do Prado Leite,et al.  The world?s a stage: a survey on requirements engineering using a real-life case study , 1999, J. Braz. Comput. Soc..

[82]  Steve Easterbrook,et al.  Communication Problems in Requirements Engineering: A Field Study , 1996 .

[83]  Colin Potts,et al.  Invented requirements and imagined customers: requirements engineering for off-the-shelf software , 1995, Proceedings of 1995 IEEE International Symposium on Requirements Engineering (RE'95).

[84]  W. W. Royce,et al.  Managing the development of large software systems: concepts and techniques , 1987, ICSE '87.

[85]  S. H. Zanakis,et al.  A Monte Carlo investigation of incomplete pairwise comparison matrices in AHP , 1997 .

[86]  Walter F. Tichy,et al.  Hints for Reviewing Empirical Work in Software Engineering , 2000, Empirical Software Engineering.

[87]  Andy J. Nolan,et al.  Learning from Success , 1999, IEEE Softw..

[88]  Ian Sommerville,et al.  Improving Market-Driven RE Processes , 1999 .

[89]  Marjo Kauppinen,et al.  Empirical Evaluation of Two Requirements Prioritization Methods in Product Development Projects , 2004, EuroSPI.

[90]  Frank Moisiadis,et al.  The fundamentals of prioritising requirements , 2002 .

[91]  Bill Curtis,et al.  A field study of the software design process for large systems , 1988, CACM.

[92]  Marjo Kauppinen,et al.  Suitability of requirements prioritization methods for market-driven software product development , 2006, Softw. Process. Improv. Pract..

[93]  Steven Sawyer,et al.  Packaged software: implications of the differences from custom approaches to software development , 2000, Eur. J. Inf. Syst..