Empirical research in requirements engineering: trends and opportunities

Requirements engineering (RE) being a foundation of software development has gained a great recognition in the recent era of prevailing software industry. A number of journals and conferences have published a great amount of RE research in terms of various tools, techniques, methods, and frameworks, with a variety of processes applicable in different software development domains. The plethora of empirical RE research needs to be synthesized to identify trends and future research directions. To represent a state-of-the-art of requirements engineering, along with various trends and opportunities of empirical RE research, we conducted a systematic mapping study to synthesize the empirical work done in RE. We used four major databases IEEE, ScienceDirect, SpringerLink and ACM and Identified 270 primary studies till the year 2012. An analysis of the data extracted from primary studies shows that the empirical research work in RE is on the increase since the year 2000. The requirements elicitation with 22 % of the total studies, requirements analysis with 19 % and RE process with 17 % are the major focus areas of empirical RE research. Non-functional requirements were found to be the most researched emerging area. The empirical work in the sub-area of requirements validation and verification is little and has a decreasing trend. The majority of the studies (50 %) used a case study research method followed by experiments (28 %), whereas the experience reports are few (6 %). A common trend in almost all RE sub-areas is about proposing new interventions. The leading intervention types are guidelines, techniques and processes. The interest in RE empirical research is on the rise as whole. However, requirements validation and verification area, despite its recognized importance, lacks empirical research at present. Furthermore, requirements evolution and privacy requirements also have little empirical research. These RE sub-areas need the attention of researchers for more empirical research. At present, the focus of empirical RE research is more about proposing new interventions. In future, there is a need to replicate existing studies as well to evaluate the RE interventions in more real contexts and scenarios. The practitioners’ involvement in RE empirical research needs to be increased so that they share their experiences of using different RE interventions and also inform us about the current requirements-related challenges and issues that they face in their work.

[1]  Betty H. C. Cheng,et al.  Research Directions in Requirements Engineering , 2007, Future of Software Engineering (FOSE '07).

[2]  Abdul Azim Abdul Ghani,et al.  Adoption of Requirements Engineering Practices in Malaysian Software Development Companies , 2010, FGIT-ASEA.

[3]  Klaus Pohl,et al.  Industry needs and research directions in requirements engineering for embedded systems , 2012, Requirements Engineering.

[4]  Jeffrey C. Carver,et al.  A systematic literature review to identify and classify software requirement errors , 2009, Inf. Softw. Technol..

[5]  Salma Imtiaz,et al.  Causes of requirement change - A systematic literature review , 2012, EASE.

[6]  A. Herrmann,et al.  Requirements Prioritization Based on Benefit and Cost Prediction: An Agenda for Future Research , 2008, 2008 16th IEEE International Requirements Engineering Conference.

[7]  Fei Peng,et al.  Understanding Chinese Characteristics of Requirements Engineering , 2009, 2009 17th IEEE International Requirements Engineering Conference.

[8]  Ricardo Melo Bastos,et al.  Systematic Review of Requirements Reuse , 2009, SEKE.

[9]  Fei Peng,et al.  Why Requirements Engineering Fails: A Survey Report from China , 2010, 2010 18th IEEE International Requirements Engineering Conference.

[10]  A. Ehrenberg,et al.  The Design of Replicated Studies , 1993 .

[11]  Alain Abran,et al.  Guide to the Software Engineering Body of Knowledge : 2004 Version , 2005 .

[12]  Bashar Nuseibeh,et al.  Requirements engineering: a roadmap , 2000, ICSE '00.

[13]  Daniela Cruzes,et al.  Research synthesis in software engineering: A tertiary study , 2011, Inf. Softw. Technol..

[14]  Rong Peng,et al.  DRE-specific Wikis for Distributed Requirements Engineering: A Review , 2012, 2012 19th Asia-Pacific Software Engineering Conference.

[15]  George Valença,et al.  Accepted Manuscript Requirements Engineering for Software Product Lines: a Systematic Literature Review Accepted Manuscript Requirements Engineering for Software Product Lines: a Systematic Literature Review Accepted Manuscript , 2022 .

[16]  David Budgen,et al.  Evaluation and assessment in software engineering , 1997, J. Syst. Softw..

[17]  Daniel Amyot,et al.  User Requirements Notation: The First Ten Years, The Next Ten Years (Invited Paper) , 2011, J. Softw..

[18]  Giuseppe Lami,et al.  An Empirical Study on the Impact of Automation on the Requirements Analysis Process , 2007, Journal of Computer Science and Technology.

[19]  Liming Zhu,et al.  Preliminary results of a systematic review on requirements evolution , 2012, EASE.

[20]  Daniela Cruzes,et al.  Recommended Steps for Thematic Synthesis in Software Engineering , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[21]  Muneera Bano,et al.  Evidence in Requirements Engineering: A Systematic Literature Review Protocol , 2011, ICSEA 2011.

[22]  Tony Gorschek,et al.  Technology transfer decision support in requirements engineering research: a systematic review of REj , 2009, Requirements Engineering.

[23]  Pearl Brereton,et al.  An Evaluation of Quality Checklist Proposals - A participant-observer case study , 2009, EASE.

[24]  F. Michel Future of Software Engineering and Multiagent Systems , 2008 .

[25]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[26]  Roel Wieringa,et al.  The methodological soundness of requirements engineering papers: a conceptual framework and two case studies , 2006, Requirements Engineering.

[27]  Greg Wilson,et al.  Requirements in the wild: How small companies do it , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[28]  Gregory V. Wilson,et al.  On the difficulty of replicating human subjects studies in software engineering , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[29]  Naveed Ikram,et al.  Evidence in software architecture, a systematic literature review , 2013, EASE '13.

[30]  Klaas Sikkel,et al.  A systematic mapping study on empirical evaluation of software requirements specifications techniques , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[31]  Torgeir Dingsøyr,et al.  Knowledge management in software engineering: A systematic review of studied concepts, findings and research methods used , 2008, Inf. Softw. Technol..

[32]  Fabio Q. B. da Silva,et al.  The Effect of Reasoning Strategies on Success in Early Learning of Programming: Lessons Learned from an External Experiment Replication , 2010, EASE.

[33]  Ghassan Beydoun,et al.  A field study of the requirements engineering practice in Australian software industry , 2007, Requirements Engineering.

[34]  Natalia Juristo Juzgado,et al.  Is the European Industry Moving toward Solving Requirements Engineering Problems? , 2002, IEEE Softw..

[35]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2000, The Kluwer International Series in Software Engineering.

[36]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[37]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[38]  Natalia Juristo Juzgado,et al.  Effectiveness of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic Review , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[39]  Carolyn B. Seaman,et al.  Qualitative Methods in Empirical Studies of Software Engineering , 1999, IEEE Trans. Software Eng..

[40]  Joaquín Nicolás,et al.  On the generation of requirements specifications from software engineering models: A systematic literature review , 2009, Inf. Softw. Technol..

[41]  Oscar Pastor,et al.  Empirical research methodologies and studies in Requirements Engineering: How far did we come? , 2014, J. Syst. Softw..

[42]  Roel Wieringa,et al.  Requirements engineering paper classification and evaluation criteria: a proposal and a discussion , 2005, Requirements Engineering.

[43]  Pearl Brereton,et al.  Evidence relating to Object-Oriented software design: A survey , 2007, ESEM 2007.

[44]  Per Runeson,et al.  Checklists for Software Engineering Case Study Research , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[45]  S. Schmidt Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences , 2009 .

[46]  Kimberly S. Wasson Requirements Metrics : Scaling Up , 2003 .

[47]  Jens von Pilgrim,et al.  A survey of traceability in requirements engineering and model-driven development , 2010, Software & Systems Modeling.

[48]  S. K. Saha,et al.  A systematic review on creativity techniques for requirements engineering , 2012, 2012 International Conference on Informatics, Electronics & Vision (ICIEV).

[49]  Ivan A. Garcia,et al.  A systematic literature review of stakeholder identification methods in requirements elicitation , 2012, J. Syst. Softw..

[50]  Natalia Juristo Juzgado,et al.  A Quantitative Assessment of Requirements Engineering Publications - 1963-2006 , 2007, REFSQ.

[51]  Mahvish Khurum,et al.  Software Requirements Triage and Selection: State-of-the-Art and State-of-Practice , 2012, 2012 19th Asia-Pacific Software Engineering Conference.

[52]  Mikio Suzuki,et al.  A Model and Architecture of REBOK (Requirements Engineering Body of Knowledge) and Its Evaluation , 2010, 2010 Asia Pacific Software Engineering Conference.

[53]  Per Runeson,et al.  Can we evaluate the quality of software engineering experiments? , 2010, ESEM '10.

[54]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007 .

[55]  Joaquín Nicolás,et al.  Risks and Safeguards for the Requirements Engineering Process in Global Software Development , 2009, 2009 Fourth IEEE International Conference on Global Software Engineering.

[56]  Silvia Mara Abrahão,et al.  Requirements Engineering in the Development of Multi-Agent Systems: A Systematic Review , 2009, IDEAL.

[57]  Andy M. Connor,et al.  Requirements Engineering Current Practice and Capability in Small and Medium Software Development Enterprises in New Zealand , 2011, 2011 Ninth International Conference on Software Engineering Research, Management and Applications.

[58]  Cheng Zhang,et al.  Preliminary Reporting Guidelines for Experience Papers , 2009, EASE.

[59]  Alan M. Davis,et al.  Requirements Researchers: Do We Practice What We Preach? , 2002, Requirements Engineering.

[60]  Martin Höst,et al.  Managing Quality Requirements: A Systematic Review , 2010, 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications.

[61]  Tore Dybå,et al.  Evidence-based software engineering , 2016, Perspectives on Data Science for Software Engineering.