Multi-Objective Search-based Requirements Selection and Optimisation

Most software product developments are iterative and incremental processes that are seldom completed in a single release. It is critical but challenging to select the requirements from a large number of candidates for the achievement of overall busi- ness goal and the goals of multiple stakeholders, each of whom may have competing and often conflicting priorities. This thesis argues that search-based techniques can be applied to the optimisation problem during the requirements selection and analysis phase for release planning problem. Search-based techniques offer significant advantages; they can be used to seek robust, scalable solutions, to investigate trade-offs, to yield insight and to provide feedback explaining choices to the decision maker. In the thesis, a Search-based Requirements Selection and Optimisation Framework is proposed that includes search spaces, representations, solution processes and em- pirical studies. This framework formulates the requirements selection problem as an optimisation problem and allows multi-objective search-based techniques to be used in order to provide optimal or near optimal solutions and to find a suitable balance between priorities in different contexts. The thesis reports the results of experiments using different multi-objective evo- lutionary optimisation algorithms with real world data sets as well as synthetic data sets in three studies of the applications of this framework: Value/Cost Trade- off in Requirements Selection, Requirements Interaction Management and Multi- Stakeholder Requirements Analysis and Optimisation. Empirical validation includes a statistical analysis of the performance of the algorithms as well as simple graph- ical methods to visualise the discovered solutions in the multi-dimensional solution space. Though these visualisations are not novel in themselves, the thesis is the first to use them for visualisation of requirements optimisation spaces.

[1]  Kazuhisa Kawai,et al.  Developed Computerized Tools Based on Mental Models for Creativity Support , 2007 .

[2]  William N. Robinson,et al.  Requirements interaction management , 2003, CSUR.

[3]  Sjaak Brinkkemper,et al.  Flexible Release Composition using Integer Linear Programming , 2004 .

[4]  Martin S. Feather,et al.  4.6.1 Combining Heuristic Search, Visualization and Data Mining for Exploration of System Design Spaces , 2004 .

[5]  Felix B. Tan,et al.  The Repertory Grid Technique: A Method for the Study of Cognition in Information Systems , 2002, MIS Q..

[6]  Vittorio Cortellessa,et al.  Automated Selection of Software Components Based on Cost/Reliability Tradeoff , 2006, EWSA.

[7]  Günther Ruhe,et al.  Hybrid Intelligence in Software Release Planning , 2004, Int. J. Hybrid Intell. Syst..

[8]  Kenneth W. Kolence The software empiricist , 1973, PERV.

[9]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[10]  Joachim Karlsson,et al.  A Cost-Value Approach for Prioritizing Requirements , 1997, IEEE Softw..

[11]  Ian F. Alexander,et al.  A Taxonomy of Stakeholders: Human Roles in System Development , 2005, Int. J. Technol. Hum. Interact..

[12]  Beatrice Alenljung,et al.  Envisioning a Future Decision Support System for Requirements Engineering : A Holistic and Human-centred Perspective , 2008 .

[13]  Stephan Jacobs Introducing measurable quality requirements: a case study , 1999, Proceedings IEEE International Symposium on Requirements Engineering (Cat. No.PR00188).

[14]  Xin Yao,et al.  A New Multi-objective Evolutionary Optimisation Algorithm: The Two-Archive Algorithm , 2006, 2006 International Conference on Computational Intelligence and Security.

[15]  Sjaak Brinkkemper,et al.  Determination of the Next Release of a Software Product: an Approach using Integer Linear Programming , 2005, CAiSE Short Paper Proceedings.

[16]  Nigel Shadbolt,et al.  A comparison of sorting techniques in knowledge acquisition , 1992 .

[17]  Yuanyuan Zhang,et al.  “Fairness Analysis” in Requirements Assignments , 2008, 2008 16th IEEE International Requirements Engineering Conference.

[18]  Günther Ruhe,et al.  Lightweight Replanning of Software Product Releases , 2006, 2006 International Workshop on Software Product Management (IWSPM'06 - RE'06 Workshop).

[19]  J. Lockerbie,et al.  REDEPEND: Extending i* Modelling into Requirements Processes , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[20]  Suzanne Robertson,et al.  Mastering the Requirements Process , 1999 .

[21]  William N. Robinson,et al.  Automated Support for Requirements Negotiation , 2002 .

[22]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[23]  Pamela Zave,et al.  An Operational Approach to Requirements Specification for Embedded Systems , 1982, IEEE Transactions on Software Engineering.

[24]  Camille Salinesi,et al.  Choosing a Prioritization Method-Case of IS Security Improvement , 2006, CAiSE Forum.

[25]  M. Kendall Rank Correlation Methods , 1949 .

[26]  Jyrki Kontio,et al.  A case study in applying a systematic method for COTS selection , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[27]  Bashar Nuseibeh,et al.  Expressing the relationships between multiple views in requirements specification , 1993, ICSE '93.

[28]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[29]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[30]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[31]  Nigel Shadbolt,et al.  Laddering: technique and tool use in knowledge acquisition , 1994 .

[32]  Mark Harman,et al.  The Current State and Future of Search Based Software Engineering , 2007, Future of Software Engineering (FOSE '07).

[33]  Jacob L. Cybulski,et al.  Requirements Classification and Reuse: Crossing Domain Boundaries , 2000, ICSR.

[34]  P Beynon-Davies,et al.  Rapid application development (RAD): an empirical review , 1999 .

[35]  Günther Ruhe,et al.  A systematic approach for solving the wicked problem of software release planning , 2007, Soft Comput..

[36]  Carlos A. Coello Coello,et al.  Evolutionary multi-objective optimization: a historical view of the field , 2006, IEEE Comput. Intell. Mag..

[37]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[38]  Martin J. Oates,et al.  The Pareto Envelope-Based Selection Algorithm for Multi-objective Optimisation , 2000, PPSN.

[39]  Stephen K. Probert Requirements Engineering, Soft Systems Methodology and Workforce Empowerment , 1999, Requirements Engineering.

[40]  Neil A. M. Maiden,et al.  Integrating creativity into requirements processes: experiences with an air traffic management system , 2005, 13th IEEE International Conference on Requirements Engineering (RE'05).

[41]  Tim Menzies,et al.  Converging on the optimal attainment of requirements , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[42]  Mark Harman,et al.  Search-based software engineering , 2001, Inf. Softw. Technol..

[43]  Barry W. Boehm,et al.  Software Requirements Negotiation and Renegotiation Aids: A Theory-W Based Spiral Approach , 1995, 1995 17th International Conference on Software Engineering.

[44]  Beatrice Alenljung,et al.  Portraying the practice of decision-making in requirements engineering: a case of large scale bespoke development , 2008, Requirements Engineering.

[45]  Björn Regnell,et al.  Requirements lifecycle management and release planning in market-driven requirements engineering processes , 2000, Proceedings 11th International Workshop on Database and Expert Systems Applications.

[46]  Ivica Crnkovic,et al.  Experimenting the Automated Selection of COTS Components Based on Cost and System Requirements , 2008, J. Univers. Comput. Sci..

[47]  Des Greer,et al.  Software release planning: an evolutionary and iterative approach , 2004, Inf. Softw. Technol..

[48]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[49]  Peter Checkland,et al.  Soft Systems Methodology in Action , 1990 .

[50]  Victor J. Rayward-Smith,et al.  The next release problem , 2001, Inf. Softw. Technol..

[51]  Martin Fowler,et al.  Patterns of Enterprise Application Architecture , 2002 .

[52]  Patrik Berander,et al.  Hierarchical Cumulative Voting (hcv) - Prioritization of Requirements in Hierarchies , 2006, Int. J. Softw. Eng. Knowl. Eng..

[53]  Des Greer,et al.  Quantitative studies in software release planning under risk and resource constraints , 2003, 2003 International Symposium on Empirical Software Engineering, 2003. ISESE 2003. Proceedings..

[54]  Neil A. M. Maiden,et al.  CREWS-SAVRE: Scenarios for Acquiring and Validating Requirements , 1998, Automated Software Engineering.

[55]  Thomas L. Saaty,et al.  Multicriteria Decision Making: The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation , 1990 .

[56]  Mark Harman,et al.  Search Based Approaches to Component Selection and Prioritization for the Next Release Problem , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[57]  Claes Wohlin,et al.  An evaluation of methods for prioritizing software requirements , 1998, Inf. Softw. Technol..

[58]  Günther Ruhe,et al.  Intelligent Support for Software Release Planning , 2004, PROFES.

[59]  Stephen Fickas,et al.  Goal-Directed Requirements Acquisition , 1993, Sci. Comput. Program..

[60]  Leslie Lamport,et al.  The temporal logic of actions , 1994, TOPL.

[61]  Wolfgang Emmerich,et al.  The future of requirements management tools , 2000 .

[62]  Joachim Karlsson,et al.  Software requirements prioritizing , 1996, Proceedings of the Second International Conference on Requirements Engineering.

[63]  Sjaak Brinkkemper,et al.  Software product release planning through optimization and what-if analysis , 2008, Inf. Softw. Technol..

[64]  Karl E. Wiegers Automating Requirements Management 1 , 1999 .

[65]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .

[66]  Pär Carlshamre,et al.  Release Planning in Market-Driven Software Product Development: Provoking an Understanding , 2002, Requirements Engineering.

[67]  Jawed I. A. Siddiqi,et al.  Requirements Engineering: The Emerging Wisdom , 1996, IEEE Softw..

[68]  Danilo Beuche,et al.  Using Requirements Management Tools in Software Product Line Engineering: The State of the Practice , 2007 .

[69]  Betty H. C. Cheng,et al.  Research Directions in Requirements Engineering , 2007, Future of Software Engineering (FOSE '07).

[70]  Joseph A. Goguen,et al.  Techniques for requirements elicitation , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.

[71]  Alistair G. Sutcliffe,et al.  Validating functional system requirements with scenarios , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[72]  Alan M. Davis,et al.  Requirements elicitation and elicitation technique selection: model for two knowledge-intensive software development processes , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[73]  Bashar Nuseibeh,et al.  Requirements engineering: a roadmap , 2000, ICSE '00.

[74]  David Chenho Kung,et al.  Software requirements and acceptance testing , 1997, Ann. Softw. Eng..

[75]  Sheldon H. Jacobson,et al.  A class of convergent generalized hill climbing algorithms , 2002, Appl. Math. Comput..

[76]  Sjaak Brinkkemper,et al.  Integrated Requirement Selection and Scheduling for the Release Planning of a Software Product , 2007, REFSQ.

[77]  Björn Regnell,et al.  A market-driven requirements engineering process: Results from an industrial process improvement programme , 1998, Requirements Engineering.

[78]  Jyrki Kontio,et al.  OTSO: a systematic process for reusable software component selection , 1995 .

[79]  Karl E. Wiegers,et al.  Software Requirements , 1999 .

[80]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[81]  P. Checkland Soft Systems Methodology: A Thirty Year Retrospective a , 2000 .

[82]  Vittorio Cortellessa,et al.  An optimization framework for "build-or-buy" decisions in software architecture , 2008, Comput. Oper. Res..

[83]  Dietmar Pfahl,et al.  Quantitative WinWin: a new method for decision support in requirements negotiation , 2002, SEKE '02.

[84]  Yuanyuan Zhang,et al.  Search Based Requirements Optimisation: Existing Work and Challenges , 2008, REFSQ.

[85]  Ed Morris,et al.  Workshop on COTS-Based Systems , 1997 .

[86]  A. M. Turing,et al.  Checking a large routine , 1989 .

[87]  Tom Gilb,et al.  Competitive Engineering: A Handbook For Systems Engineering, Requirements Engineering, And Software Engineering Using Planguage , 2005 .

[88]  Neil A. M. Maiden,et al.  Making Mobile Requirements Engineering Tools Usable and Useful , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[89]  Marie-Claude Gaudel,et al.  Structuring and Modularizing Algebraic Specifications: The PLUSS Specification Language, Evolutions and Perspectives , 1992, STACS.

[90]  Neil A. M. Maiden,et al.  ACRE: selecting methods for requirements acquisition , 1996, Softw. Eng. J..

[91]  Ian Sommerville,et al.  Social analysis in the requirements engineering process: from ethnography to method , 1999, Proceedings IEEE International Symposium on Requirements Engineering (Cat. No.PR00188).

[92]  Eric S. K. Yu,et al.  Towards modelling and reasoning support for early-phase requirements engineering , 1997, Proceedings of ISRE '97: 3rd IEEE International Symposium on Requirements Engineering.

[93]  Marco Dorigo,et al.  Optimization, Learning and Natural Algorithms , 1992 .

[94]  Neil A. M. Maiden,et al.  Modelling Component Dependencies to Inform Their Selection , 2003, ICCBSS.

[95]  Richard Bentley,et al.  Integrating ethnography into the requirements engineering process , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.

[96]  Mark Gershon,et al.  Techniques for multiobjective decision making in systems management , 1986 .

[97]  Soren Lauesen,et al.  Software Requirements: Styles & Techniques , 2002 .

[98]  E. F. Codd,et al.  Extending the database relational model to capture more meaning , 1979, ACM Trans. Database Syst..

[99]  Ljerka Beus-Dukic,et al.  Discovering Requirements: How to Specify Products and Services , 2009 .

[100]  Mark Harman,et al.  An empirical study of the robustness of two module clustering fitness functions , 2005, GECCO '05.

[101]  Tony Gorschek,et al.  Quality Requirements in Practice: An Interview Study in Requirements Engineering for Embedded Systems , 2009, REFSQ.

[102]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[103]  Olly Gotel,et al.  An analysis of the requirements traceability problem , 1994, Proceedings of IEEE International Conference on Requirements Engineering.

[104]  Neil A. M. Maiden,et al.  Automatically Generating Requirements from i* Models: Experiences with a Complex Airport Operations System , 2007, REFSQ.

[105]  Yuanyuan Zhang,et al.  The multi-objective next release problem , 2007, GECCO '07.

[106]  Yuanyuan Zhang,et al.  A search based approach to fairness analysis in requirement assignments to aid negotiation, mediation and decision making , 2009, Requirements Engineering.

[107]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[108]  Cornelius Ncube,et al.  Rethinking Process Guidance for Selecting Software Components , 2002, ICCBSS.

[109]  Anthony Finkelstein,et al.  Requirements engineering: a review and research agenda , 1994, Proceedings of 1st Asia-Pacific Software Engineering Conference.

[110]  Andrzej Osyczka,et al.  7 – Multicriteria optimization for engineering design , 1985 .

[111]  Cornelius Ncube,et al.  PORE : Procurement Oriented Requirements Engineering Method for the Component-Based Systems Engineering Development Paradigm , 1999 .

[112]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[113]  Bashar Nuseibeh,et al.  Coordinating distributed ViewPoints: the Anatomy of a Consistency Check , 1994 .

[114]  Balasubramaniam Ramesh,et al.  Implementing requirements traceability: a case study , 1995, Proceedings of 1995 IEEE International Symposium on Requirements Engineering (RE'95).

[115]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[116]  Ho-Won Jung,et al.  Optimizing Value and Cost in Requirements Analysis , 1998, IEEE Softw..

[117]  Armin Eberlein,et al.  COTS Selection: Past, Present, and Future , 2007, 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS'07).

[118]  Shin Yoo,et al.  Search based data sensitivity analysis applied to requirement engineering , 2009, GECCO.

[119]  Ian Sommerville,et al.  Requirements Engineering: Processes and Techniques , 1998 .

[120]  Constance L. Heitmeyer,et al.  Automated consistency checking of requirements specifications , 1996, TSEM.

[121]  Lawrence Chung,et al.  COTS-Aware Requirements Engineering and Software Architecting , 2004, Software Engineering Research and Practice.

[122]  B. Swanson,et al.  PRISM Product Examination Process for component based development , 1997, Proceedings Fifth International Symposium on Assessment of Software Tools and Technologies.

[123]  Joachim Karlsson,et al.  A systematic approach for prioritizing software requirements , 1998 .

[124]  Edward Yourdon,et al.  Object-oriented analysis , 2012 .

[125]  Neil A. M. Maiden,et al.  ART-SCENE: enhancing scenario walkthroughs with multi-media scenarios , 2004, Proceedings. 12th IEEE International Requirements Engineering Conference, 2004..

[126]  Ivica Crnkovic,et al.  COTS Selection Best Practices in Literature and in Industry , 2008, ICSR.

[127]  Mark Harman,et al.  Search--based approaches to the component selection and prioritization problem , 2006, GECCO.

[128]  Neil A. M. Maiden,et al.  Guiding parallel requirements acquisition and COTS software selection , 1999, Proceedings IEEE International Symposium on Requirements Engineering (Cat. No.PR00188).

[129]  Alan M. Davis,et al.  Operational prototyping: a new development approach , 1992, IEEE Software.

[130]  Jane Sinclair,et al.  Introduction to formal specification and Z , 1991, Prentice Hall International Series in Computer Science.

[131]  Axel van Lamsweerde,et al.  Requirements engineering in the year 00: a research perspective , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[132]  A. T. Bahill What is systems engineering , 1995 .

[133]  Günther Ruhe,et al.  The art and science of software release planning , 2005, IEEE Software.

[134]  C. A. R. Hoare,et al.  An axiomatic basis for computer programming , 1969, CACM.

[135]  Robert W. Floyd,et al.  Assigning Meanings to Programs , 1993 .

[136]  Marsha Chechik,et al.  A framework for multi-valued reasoning over inconsistent viewpoints , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.