Comparing alternatives for analyzing requirements trade-offs - In the absence of numerical data

Context: Choosing a design solution most often involves dealing with trade-offs and conflicts among requirements and design objectives. Making such trade-offs during early stages of requirements and design is challenging because costs and benefits of alternatives are often hard to quantify. Objective: The objective of this work is to develop a decision analysis method that assists in making trade-offs in the absence of quantitative data. Method: In this method, stakeholders qualitatively compare consequences of alternatives on decision criteria. We propose an algorithm that generates all possible consequences of alternatives on requirements, according to the rough qualitative comparisons that stakeholders made. The possible consequences generated by the algorithm are then analyzed by the Even Swaps Multi-Criteria Decision Analysis method to determine the best solution. The Even Swaps method is a technique developed in management science to assist in multi-criteria decision making when explicit value trade-offs are not available. Results and conclusions: Our algorithm teases out the need to accurately measure or estimate costs and benefits of alternative design solutions. The algorithm automates the Even Swap process, and reuses stakeholders' value trade-offs throughout the Even Swaps process. We applied the prototype tool in several case studies to evaluate the utility of the method. The results of case studies provide evidence that our decision aid method selects the optimum solution correctly compared to results of other similar quantitative methods, while our method does not rely on detailed numerical assessment of alternatives and importance weights of criteria.