Requirements elicitation methods based on interviews in comparison: A family of experiments

Abstract Context There are several methods to elicit requirements through interviews between an end-user and a team of software developers. The choice of the best method in this context is usually on subjective developers’ preferences instead of objective reasons. There is a lack of empirical evaluations of methods to elicit requirements that help developers to choose the most suitable one. Objective This paper designs and conducts a family of experiments to compare three methods to elicit requirements: Unstructured Interviews, where there is no specific protocol or artifacts; Joint Application Design (JAD), where each member of the development team has a specific role; Paper Prototyping, where developers contrast the requirements with the end-user through prototypes. Method The experiment is a between-subjects design with next response variables: number of requirements, time, diversity, completeness, quality and performance. The experiment consists of a maximum of 4 rounds of interviews between students that play the role of developers and an instructor that plays the role of client. Subjects had to elaborate a requirements specification document as results of the interviews. We recruited 167 subjects in 4 replications in 3 years. Subjects were gathered in development teams of 6 developers at most, and each team was an experimental unit. Results We found some significant differences. Paper Prototyping yields the best results to elicit as many requirements as possible, JAD requires the highest time to report the requirements and the least overlapping, and Unstructured Interviews yields the highest overlapping and the lowest time to report the requirements. Conclusions Paper Prototyping is the most suitable for eliciting functional requirements, JAD is the most suitable for non-functional requirements and to avoid overlapping, Unstructured Interviews is the fastest but with poor quality in the results.

[1]  Klaus Schmid,et al.  Effective Requirements Elicitation in Product Line Application Engineering - An Experiment , 2013, REFSQ.

[2]  G. Raju,et al.  Requirements Elicitation Using Paper Prototype , 2010, FGIT-ASEA.

[3]  Julio Cesar Sampaio do Prado Leite,et al.  On Non-Functional Requirements in Software Engineering , 2009, Conceptual Modeling: Foundations and Applications.

[4]  Fatemeh Zahedi,et al.  Enhancing Analysts' Mental Models for Improving Requirements Elicitation: A Two-stage Theoretical Framework and Empirical Results , 2016, J. Assoc. Inf. Syst..

[5]  Kurt Schneider,et al.  Videos vs. Use Cases: Can Videos Capture More Requirements under Time Pressure? , 2010, REFSQ.

[6]  Atul Gupta,et al.  A systematic literature review of use case specifications research , 2015, Inf. Softw. Technol..

[7]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[8]  Natalia Juristo Juzgado,et al.  Systematizing requirements elicitation technique selection , 2014, Inf. Softw. Technol..

[9]  Glenn J. Browne,et al.  An Empirical Investigation of User Requirements Elicitation: Comparing the Effectiveness of Prompting Techniques , 2001, J. Manag. Inf. Syst..

[10]  Franz Lehner,et al.  Requirements Engineering as a Success Factor in Software Projects , 2001, IEEE Softw..

[11]  Seiichi Komiya,et al.  A method to navigate interview-driven software requirements elicitation work: an efficacy evaluation of the method from the viewpoint of efficiency , 2010 .

[12]  Janice Singer,et al.  Ethical Issues in Empirical Studies of Software Engineering , 2002, IEEE Trans. Software Eng..

[13]  Anitha S. Pillai,et al.  Requirements elicitation issues and project performance: A test of a contingency model , 2015, 2015 Science and Information Conference (SAI).

[14]  Jussi Tohka,et al.  How Many Is Enough? Effect of Sample Size in Inter-Subject Correlation Analysis of fMRI , 2016, Comput. Intell. Neurosci..

[15]  Catarina Gralha.  Evaluation of Requirements Models , 2016, 2016 IEEE 24th International Requirements Engineering Conference (RE).

[16]  Naveed Ikram,et al.  Empirical research in requirements engineering: trends and opportunities , 2016, Requirements Engineering.

[17]  Klaus Pohl,et al.  Requirements Engineering - Fundamentals, Principles, and Techniques , 2010 .

[18]  Ian Chai,et al.  A Controlled Experiment on Requirements Elicitation in Electronic Markets , 2017, IC4E '17.

[19]  Mary Beth Rosson,et al.  Effectiveness of elicitation techniques in distributed requirements engineering , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[20]  Guilherme Horta Travassos,et al.  Is a Picture worth a Thousand Words?: A Comparative Analysis of Using Textual and Graphical Approaches to Specify Use Cases , 2016, SBES.

[21]  Uolevi Nikula,et al.  Requirements Elicitation Using a Combination of Prototypes and Scenarios , 2001, WER.

[22]  Monique Snoeck,et al.  Trust in requirements elicitation: how does it build, and why does it matter to requirements engineers? , 2017, SAC.

[23]  P. D. D. Dominic,et al.  Assessment and evaluation of requirements elicitation techniques using analysis determination requirements framework , 2014, 2014 International Conference on Computer and Information Sciences (ICCOINS).

[24]  Colin J. Neill,et al.  Requirements Engineering: The State of the Practice , 2003, IEEE Softw..

[25]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[26]  Muneera Bano,et al.  Interview Review: An Empirical Study on Detecting Ambiguities in Requirements Elicitation Interviews , 2018, REFSQ.

[27]  Erran Carmel,et al.  PD and joint application design: a transatlantic comparison , 1993, CACM.

[28]  Alain F. Zuur,et al.  A protocol for data exploration to avoid common statistical problems , 2010 .

[29]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[30]  Peter Sawyer,et al.  On the Effectiveness of Abstraction Identification in Requirements Engineering , 2010, 2010 18th IEEE International Requirements Engineering Conference.

[31]  John A. van der Poll,et al.  Towards a Methodology to Elicit Tacit Domain Knowledge from Users , 2007 .

[32]  Didar Zowghi,et al.  Interview Review: Detecting Latent Ambiguities to Improve the Requirements Elicitation Process , 2017, 2017 IEEE 25th International Requirements Engineering Conference (RE).

[33]  Sajjad Mahmood,et al.  Software Requirements Elicitation -- A Controlled Experiment to Measure the Impact of a Native Natural Language , 2013, 2013 IEEE 37th Annual Computer Software and Applications Conference.

[34]  Sjaak Brinkkemper,et al.  Gamified Requirements Engineering: Model and Experimentation , 2016, REFSQ.

[35]  Anna Perini,et al.  Speech-acts based analysis for requirements discovery from online discussions , 2019, Inf. Syst..

[36]  Natalia Juristo Juzgado,et al.  Effect of Domain Knowledge on Elicitation Effectiveness: An Internally Replicated Controlled Experiment , 2016, IEEE Transactions on Software Engineering.

[37]  Irit Hadar,et al.  The role of domain knowledge in requirements elicitation via interviews: an exploratory study , 2012, Requirements Engineering.

[38]  Laurie A. Williams,et al.  Identifying the implied: Findings from three differentiated replications on the use of security requirements templates , 2016, Empirical Software Engineering.

[39]  Didar Zowghi,et al.  Requirements Elicitation: A Survey of Techniques, Approaches, and Tools , 2005 .

[40]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[41]  Anitha S. Pillai,et al.  The Effects of Requirements Elicitation Issues on Software Project Performance: An Empirical Analysis , 2014, REFSQ.

[42]  Jouni Similä,et al.  Utilizing online serious games to facilitate distributed requirements elicitation , 2015, J. Syst. Softw..

[43]  P. Dalgaard,et al.  Analysis of variance and the Kruskal–Wallis test , 2008 .

[44]  Dean Leffingwell,et al.  Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise , 2011 .