Commonalities and differences between requirements engineering tools: A quantitative approach

System and software developers are concerned to gain insight into how current requirements engineering (RE) tools support processes. There is an important number of RE tools currently available on the market but, unfortunately, existing RE tool lists do not usually provide detailed and precise information about the tools they catalogue. In this paper, we study and compare current RE tools in the quest to answer the following research question: What level of variation, in terms of functionality, is observable in state-of-practice RE tools? A 188-item survey was designed, aimed at major tool vendors worldwide and based principally on the features covered by the ISO/IEC TR 24766:2009. Extensive data obtained from 29 participants was used to classify and group the RE tools, based on their capabilities. First of all, an inter-rater reliability analysis was performed to ensure the trustworthiness of the data. Descriptive statistics, hierarchical cluster analysis and statistical hypothesis testing were then applied. The tool scores for each candidate were calculated. A total of three clusters were identified. Statistically significant differences in coverage of features among these groups came to light. Our findings can help practitioners to decide which tool is the most suitable among several alternatives, according to their particular needs.

[1]  J.S. Reel,et al.  Critical Success Factors in Software Projects , 1999, IEEE Softw..

[2]  Michalis Vazirgiannis,et al.  On Clustering Validation Techniques , 2001, Journal of Intelligent Information Systems.

[3]  Axel van Lamsweerde,et al.  Requirements engineering in the year 00: a research perspective , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[4]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[5]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[6]  Magne Jørgensen,et al.  Can you Trust a Single Data Source Exploratory Software Engineering Case Study? , 2002, Empirical Software Engineering.

[7]  Casper Lassenius,et al.  Perceived causes of software project failures - An analysis of their relationships , 2014, Inf. Softw. Technol..

[8]  Michael M. Sokal,et al.  Manufacturing Knowledge: A History of the Hawthorne Experiments , 1991 .

[9]  Jaap Gordijn,et al.  What is This Science Called Requirements Engineering? , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[10]  Christof Ebert,et al.  ReqIF: Seamless Requirements Interchange Format between Business Partners , 2012, IEEE Software.

[11]  Barbara Kitchenham,et al.  DESMET: a methodology for evaluating software engineering methods and tools , 1997 .

[12]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[13]  Capers Jones,et al.  Why software fails , 1996 .

[14]  Rafael Prikladnicki,et al.  Managing Global Software Engineering: A Comparative Analysis of Offshore Outsourcing and the Internal Offshoring of Software Development , 2012, Inf. Syst. Manag..

[15]  Franz Lehner,et al.  Requirements Engineering as a Success Factor in Software Projects , 2001, IEEE Softw..

[16]  Lawrence D. Pohlmann,et al.  Customer‐Centered Products – Creating Successful Products through Smart Requirements Management , 2001 .

[17]  Christof Ebert,et al.  Improving the Exchange of Requirements and Specifications between Business Partners , 2009, 2009 17th IEEE International Requirements Engineering Conference.

[18]  Bashar Nuseibeh,et al.  Requirements engineering: a roadmap , 2000, ICSE '00.

[19]  Aurora Vizcaíno,et al.  Requirements Engineering Tools , 2011, IEEE Software.

[20]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[21]  Robert L. Glass Software Runaways: Monumental Software Disasters , 1997 .

[22]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[23]  William M. K. Trochim,et al.  Research methods knowledge base , 2001 .

[24]  Matthias Weber,et al.  Requirements for requirements management tools , 2004, Proceedings. 12th IEEE International Requirements Engineering Conference, 2004..

[25]  Joaquín Nicolás,et al.  Risks and Safeguards for the Requirements Engineering Process in Global Software Development , 2009, 2009 Fourth IEEE International Conference on Global Software Engineering.

[26]  Stefan Biffl,et al.  Value-Based Selection of Requirements Engineering Tool Support , 2006, 32nd EUROMICRO Conference on Software Engineering and Advanced Applications (EUROMICRO'06).

[27]  Sudipto Guha,et al.  CURE: an efficient clustering algorithm for large databases , 1998, SIGMOD '98.

[28]  Jarmo J. Ahonen,et al.  Software engineering projects may fail before they are started: Post-mortem analysis of five cancelled projects , 2010, J. Syst. Softw..

[29]  Richard Burns,et al.  Business Research Methods and Statistics Using SPSS , 2008 .

[30]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[31]  Victor R. Basili,et al.  The TAME Project: Towards Improvement-Oriented Software Environments , 1988, IEEE Trans. Software Eng..

[32]  Lee J. White Editorial: The importance of empirical work for software engineering papers , 2002, Softw. Test. Verification Reliab..

[33]  Aurora Vizcaíno,et al.  Requirements engineering tools: Capabilities, survey and assessment , 2012, Inf. Softw. Technol..

[34]  Forrest Shull,et al.  Patterns in Effective Distributed Software Development , 2010, IEEE Software.

[35]  James D. Herbsleb,et al.  Global Software Engineering: The Future of Socio-technical Coordination , 2007, Future of Software Engineering (FOSE '07).

[36]  R. Lyman Ott.,et al.  An introduction to statistical methods and data analysis , 1977 .

[37]  Betty H. C. Cheng,et al.  Research Directions in Requirements Engineering , 2007, Future of Software Engineering (FOSE '07).

[38]  Christof Ebert Global Software and IT: A Guide to Distributed Development, Projects, and Outsourcing , 2011 .

[39]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[40]  June M. Verner,et al.  Why did your project fail? , 2009, Commun. ACM.

[41]  Khaled El Emam,et al.  A Replicated Survey of IT Software Project Failures , 2008, IEEE Software.

[42]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[43]  R.N. Charette,et al.  Why software fails [software failure] , 2005, IEEE Spectrum.

[44]  Moshe Y. Vardi Globalization and Offshoring of Software revisited , 2010, Commun. ACM.

[45]  M. Norusis IBM SPSS Statistics 19 Statistical Procedures Companion , 2011 .

[46]  Lionel C. Briand,et al.  A practical guide for using statistical tests to assess randomized algorithms in software engineering , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[47]  Leslie P. Willcocks,et al.  The Handbook of Global Outsourcing and Offshoring , 2009 .

[48]  David J. Livingstone,et al.  A Practical Guide to Scientific Data Analysis , 2009 .

[49]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[50]  J. H. Ward Hierarchical Grouping to Optimize an Objective Function , 1963 .

[51]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.