Systematic literature reviews in software engineering: Preliminary results from interviews with researchers

Systematic Literature Reviews (SLRs) have been gaining significant attention from software engineering researchers since 2004. Several researchers have reported their experiences of and lessons learned from applying systematic reviews to different subject matters in software engineering. However, there has been no attempt at independently exploring experiences and perceptions of the practitioners of systematic reviews in order to gain an in-depth understanding of various aspects of systemic reviews as a new research methodology in software engineering. We assert that there is a need of evidence-based body of knowledge about the application of systematic reviews in software engineering. To address this need, we have started an empirical research program that aims to contribute to the growing body of knowledge about systematic reviews in software engineering. This paper reports the design, logistics, and results of the first phase empirical study carried out in this program. The results provide interesting insights into different aspects of systematic reviews based on the analysis of the data gathered from 17 interviewees with varying levels of knowledge of and experiences in systematic reviews. The findings from this study are expected to contribute to the existing knowledge about using systematic reviews and help further improve the state-of-the-practice of this research methodology in software engineering.

[1]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[2]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[3]  Paula Gomes Mian,et al.  Systematic Review in Software Engineering , 2005 .

[4]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[5]  Tore Dybå,et al.  Evidence-Based Software Engineering for Practitioners , 2005, IEEE Softw..

[6]  M. Patton,et al.  Qualitative evaluation and research methods , 1992 .

[7]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[8]  Shari Lawrence Pfleeger,et al.  Principles of survey research part 2: designing a survey , 2002, SOEN.

[9]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007 .

[10]  Shari Lawrence Pfleeger,et al.  Principles of survey research: part 1: turning lemons into lemonade , 2001, SOEN.

[11]  Carolyn B. Seaman,et al.  Qualitative Methods in Empirical Studies of Software Engineering , 1999, IEEE Trans. Software Eng..

[12]  Mark Staples,et al.  Experiences using systematic review guidelines , 2006, J. Syst. Softw..

[13]  Mark Staples,et al.  Systematic review of organizational motivations for adopting CMM-based SPI , 2008, Inf. Softw. Technol..

[14]  Per Runeson,et al.  Empirical evaluations of regression test selection techniques: a systematic review , 2008, ESEM '08.

[15]  Dietmar Pfahl,et al.  Reflections on 10 Years of Software Process Simulation Modeling: A Systematic Review , 2008, ICSP.

[16]  Tore Dybå,et al.  A systematic review of statistical power in software engineering experiments , 2006, Inf. Softw. Technol..

[17]  Guilherme Horta Travassos,et al.  Cross versus Within-Company Cost Estimation Studies: A Systematic Review , 2007, IEEE Transactions on Software Engineering.

[18]  Maria Teresa Baldassarre,et al.  Systematic Review of Statistical Process Control: An Experience Report , 2007, EASE.

[19]  Tore Dybå,et al.  Empirical studies of agile software development: A systematic review , 2008, Inf. Softw. Technol..

[20]  Tore Dybå,et al.  The Future of Empirical Methods in Software Engineering Research , 2007, Future of Software Engineering (FOSE '07).

[21]  Tore Dybå,et al.  Strength of evidence in systematic reviews in software engineering , 2008, ESEM '08.

[22]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[23]  Tore Dybå,et al.  Applying Systematic Reviews to Diverse Study Types: An Experience Report , 2007, ESEM 2007.

[24]  Guenther Ruhe,et al.  Impact Analysis of Missing Values on the Prediction Accuracy of Analogy-based Software Effort Estimation Method AQUA , 2007, ESEM 2007.